[FFmpeg-cvslog] avformat/rmdec: Fix codecdata_length overflow check

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Fri Dec 18 23:13:58 2020 +0100| [3c41d0bfd6041890b394a3e6eb2f8da92b83416b] | committer: Michael Niedermayer

avformat/rmdec: Fix codecdata_length overflow check

Fixes: signed integer overflow: 2147483647 + 64 cannot be represented in type 'int'
Fixes: 28509/clusterfuzz-testcase-minimized-ffmpeg_dem_IVR_fuzzer-6310969680723968

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=3c41d0bfd6041890b394a3e6eb2f8da92b83416b
---

 libavformat/rmdec.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavformat/rmdec.c b/libavformat/rmdec.c
index 004c62086d..ae83a8afe7 100644
--- a/libavformat/rmdec.c
+++ b/libavformat/rmdec.c
@@ -254,7 +254,7 @@ static int rm_read_audio_stream_info(AVFormatContext *s, AVIOContext *pb,
             if (version == 5)
                 avio_r8(pb);
             codecdata_length = avio_rb32(pb);
-            if(codecdata_length + AV_INPUT_BUFFER_PADDING_SIZE <= (unsigned)codecdata_length){
+            if((unsigned)codecdata_length > INT_MAX - AV_INPUT_BUFFER_PADDING_SIZE){
                 av_log(s, AV_LOG_ERROR, "codecdata_length too large\n");
                 return -1;
             }