[FFmpeg-devel] [PATCH] libavformat/avc.c annexb parsing corrupts packets

[matteo.elli at m-elli.it]

> code in libavformat/avc.c is seldom executed but it surely does when you transmux (vcodec=copy) from h264 in a ts container to h264 in an mp4/mov/mkv/avi container.
>
> The code there fails to handle annexb 00 00 01 NAL three byets prefixes properly and cuts the last byte in the previous NAL corrupting the whole frame.

please provide a reproduceable testcase

..

----------

ok I have arranged a little testcase, see these files:

http://www.m-elli.com/annexb-test.ts 

the original ts file

http://www.m-elli.com/annexb-test-ffmpeg-0.5.mp4 

the original ts file processed by ffmpeg-0.5 without patch (command "ffmpeg -i annexb-test.ts -vcodec copy -an -f mp4 annexb-test-ffmpeg-0.5.mp4")

http://www.m-elli.com/annexb-test-ffmpeg-0.5-patched.mp4 

the original ts file processed by ffmpeg-0.5 with patch (command "ffmpeg -i annexb-test.ts -vcodec copy -an -f mp4 annexb-test-ffmpeg-0.5-patched.mp4")

http://www.m-elli.com/annexb-test-nal-tree.txt 

the NAL tree generated with H264Visa from the original ts file, I have commented the first pict comparing lengths to the annexb-test-ffmpeg-0.5.mp4 file like this:

+ NAL(Pic No. in decoding order)
    + Before First Picture
        + NAL Type: AUD, Len: 2, Pos: 10(0xa)
    + Pic 1(Dec Order)
        + NAL Type: SPS, Len: 25, Pos: 19(0x13)
        + NAL Type: PPS, Len: 4, Pos: 48(0x30)
        + NAL Type: SEI, Len: 11, Pos: 56(0x38)   -> becomes len 12 (0x0c)
        + NAL Type: IDR, Len: 35837, Pos: 71(0x47)  -> becomes len 35836 (0x8bfc) -> one byte truncated !!!
        + NAL Type: IDR, Len: 37451, Pos: 35912(0x8c48)  -> becomes len 37452 (0x924C)
        + NAL Type: IDR, Len: 38904, Pos: 73371(0x11e9b) -> becomes len 38908 (0x97fc)
        + NAL Type: IDR, Len: 39717, Pos: 112279(0x1b697) -> becomes len 39716 (0x9b24) -> one byte truncated !!!
        + NAL Type: IDR, Len: 61112, Pos: 152003(0x251c3) -> becomes len 61116 (0xeebc)
        + NAL Type: IDR, Len: 31483, Pos: 213125(0x34085) -> becomes len 31489 (0x7b01)

as you can see NAL lengths are wrong, this is usually not an issue when len is greater but in slices 1 and 4 one byte is missing and the whole pict gets corrupted

matteo elli