[FFmpeg-cvslog] avformat/nutdec: Check tmp_size

[Michael Niedermayer]

ffmpeg | branch: release/4.4 | Michael Niedermayer <michael at niedermayer.cc> | Sun Apr 25 20:01:03 2021 +0200| [acfce11c48a1d19d16b9cef798e9350e44581cd5] | committer: Michael Niedermayer

avformat/nutdec: Check tmp_size

Fixes: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 29102/clusterfuzz-testcase-minimized-ffmpeg_dem_NUT_fuzzer-6739990530883584

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 1ca00b5e44f21840b608e238fa135a1aab6e576b)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=acfce11c48a1d19d16b9cef798e9350e44581cd5
---

 libavformat/nutdec.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libavformat/nutdec.c b/libavformat/nutdec.c
index fbecf71328..58a74612a4 100644
--- a/libavformat/nutdec.c
+++ b/libavformat/nutdec.c
@@ -286,6 +286,11 @@ static int decode_main_header(NUTContext *nut)
             ret = AVERROR_INVALIDDATA;
             goto fail;
         }
+        if (tmp_size < 0 || tmp_size > INT_MAX - count) {
+            av_log(s, AV_LOG_ERROR, "illegal size\n");
+            ret = AVERROR_INVALIDDATA;
+            goto fail;
+        }
 
         for (j = 0; j < count; j++, i++) {
             if (i == 'N') {