[FFmpeg-devel] [PATCH 2/7] avcodec/nuv: widen buf_size type

[Michael Niedermayer]

Fixes: signed integer overflow: 65312 * 65312 cannot be represented in type 'int'
Fixes: 20492/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_NUV_fuzzer-5740176118906880

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
---
 libavcodec/nuv.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/nuv.c b/libavcodec/nuv.c
index 7958000ae8..3ceaaac4e9 100644
--- a/libavcodec/nuv.c
+++ b/libavcodec/nuv.c
@@ -126,7 +126,7 @@ static int codec_reinit(AVCodecContext *avctx, int width, int height,
         get_quant_quality(c, quality);
     if (width != c->width || height != c->height) {
         // also reserve space for a possible additional header
-        int buf_size = height * width * 3 / 2
+        int64_t buf_size = height * (int64_t)width * 3 / 2
                      + FFMAX(AV_LZO_OUTPUT_PADDING, AV_INPUT_BUFFER_PADDING_SIZE)
                      + RTJPEG_HEADER_SIZE;
         if (buf_size > INT_MAX/8)
-- 
2.17.1