[FFmpeg-trac] #10908(avformat:new): avformat/mpegtsenc: mpegts_check_bitstream broken since 8559cce3c37b (wrong operator precedence)

[FFmpeg]

#10908: avformat/mpegtsenc: mpegts_check_bitstream broken since 8559cce3c37b (wrong
operator precedence)
-------------------------------------+-------------------------------------
             Reporter:  Sebastian    |                     Type:  defect
  Meyer                              |
               Status:  new          |                 Priority:  normal
            Component:  avformat     |                  Version:  git-
                                     |  master
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 **Summary of the bug:**
 Commit
 [https://github.com/FFmpeg/FFmpeg/commit/8559cce3c37b08e37fd208b8895a87adb8832e77
 8559cce3c37b] refactored mpegtsenc's `mpegts_check_bitstream` function and
 added a codec-id lookup table instead of checking each codec-id
 individually. In the process, a `uint8_t` bitmask was applied to the
 stream's `st->codecpar->extradata[0]` which was intended to be checked
 against the lookup table's `value` field.

 However, the commit author and the reviewers made a mistake and didn't
 notice the incorrect operator precedence of the bitwise-and and equality
 comparison.

 Beginning from this commit until today's HEAD commit on the master branch,
 muxing MPEG-TS fails for certain inputs.

 **This issue was originally noticed here:**
 https://github.com/streamlink/streamlink/issues/5876

 **How to reproduce:**
 In the issue thread linked above, there's sample data of two MPEG-TS
 streams, one video and one audio stream. Remuxing into one MPEG-TS stream
 fails due to this bug. Builds of earlier commits work fine.
 https://github.com/streamlink/streamlink/files/14537316/5876.zip

 The error messages are this:
 {{{
 [mpegts @ 0x6392afb61240] h264 bitstream error, startcode missing, size
 100902 data 00018A22
 [mpegts @ 0x6392afb61240] h264 bitstream error, startcode missing, size
 539 data 00000217
 [mpegts @ 0x6392afb61240] h264 bitstream error, startcode missing, size
 165 data 000000A1
 ...
 }}}

 Since I'm not familiar with mailing lists and since I didn't find the bug
 myself, I'm not going to send a patch. The person who found this bug in
 the linked issue thread apparently wanted to submit a patch, but hasn't
 responded since, so I decided to open this bug report instead. Maybe some
 of the responsible devs want to have a look at this again anyway.

 Thanks.
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10908>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker