[FFmpeg-trac] #9713(avcodec:new): Hardware accelerated decoding fails on M1 macs for certain videos encoded with h264

[FFmpeg]

#9713: Hardware accelerated decoding fails on M1 macs for  certain videos encoded
with h264
-------------------------------------+-------------------------------------
             Reporter:  mikffmpeg    |                    Owner:  (none)
                 Type:  defect       |                   Status:  new
             Priority:  normal       |                Component:  avcodec
              Version:  git-master   |               Resolution:
             Keywords:  avcodec;     |               Blocked By:
  m1; mac; h264                      |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
Comment (by Alex Agranovsky):

 So, I've found something interesting:
 PPS NAL unit, as it appears in the bitstream, is 4 bytes long. PPS when it
 being used by videotoolbox.c, is 3 bytes long.

 I've tested a theory, and indeed, if I add the missing last byte, 0x80, to
 PPS being used for videotoolbox initialization, previously failing streams
 work with videotoolbox on M1.

 However now I'm finding myself in H264 parser code, which I'm not overfly
 familiar with, and seeing some definite weirdness there:
 `ff_h264_decode_picture_parameter_set` is being called twice, and `gb`
 parameter first indicates the length of 5, and length of 3 in the second
 call.

 What is the expectation? Is the parsed PPS buffer supposed to preserve the
 size of PPS NALU? I'm attaching the raw error.h264 I'm working with ...
 these are the initial NAL units in that file:

 {{{
 00 00 00 02             9 - Access unit delimiter
 09 f0
 00 00 00 1b             7 - SPS
 67 4d 00 2a 9d a8 1e 00 89 f9 70 06 e0 20 20 28
 00 00 03 00 08 00 00 03 01 94 20
 00 00 00 04             8 - PPS
 68 ee 3c 80
 00 00 00 05             6 - SEI
 06 e5 01 41 80
 00 02 04 b7             5 - IDR
 65 b8 00 00 0e ab 00 cf f7 49 24 77 ...

 }}}
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9713#comment:6>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker