[FFmpeg-devel] [PATCH 2/2] avcodec/h264_parser: fix nalsize parser

[Andreas Rheinhardt]

Michael Niedermayer:
> Fixes: left shift of 16711968 by 8 places cannot be represented in type 'int'
> Fixes: 36601/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_H264_fuzzer-6581933285965824
> 
> Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
> Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
> ---
>  libavcodec/h264_parser.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/libavcodec/h264_parser.c b/libavcodec/h264_parser.c
> index d3c56cc188..22111c62a2 100644
> --- a/libavcodec/h264_parser.c
> +++ b/libavcodec/h264_parser.c
> @@ -86,7 +86,7 @@ static int h264_find_frame_end(H264ParseContext *p, const uint8_t *buf,
>              int nalsize = 0;
>              i = next_avc;
>              for (j = 0; j < p->nal_length_size; j++)
> -                nalsize = (nalsize << 8) | buf[i++];
> +                nalsize = ((unsigned)nalsize << 8) | buf[i++];
>              if (nalsize <= 0 || nalsize > buf_size - i) {
>                  av_log(logctx, AV_LOG_ERROR, "AVC-parser: nal size %d remaining %d\n", nalsize, buf_size - i);
>                  return buf_size;
> 
Makes me wonder why I never applied this:
https://patchwork.ffmpeg.org/project/ffmpeg/patch/20200529161755.9904-1-andreas.rheinhardt@gmail.com/
(Your fix would only fix the undefined behaviour, not the nonsense
logmessage (with negative sizes) one gets in this scenarion.)

- Andreas