[FFmpeg-cvslog] avcodec/agm: YUV420 without DCT needs even dimensions

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Fri Jan 10 21:30:38 2020 +0100| [a98eeb0c1e867238905ed095b48184f706adf328] | committer: Michael Niedermayer

avcodec/agm: YUV420 without DCT needs even dimensions

Fixes: out of array access
Fixes: 19892/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_AGM_fuzzer-5707525924323328

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=a98eeb0c1e867238905ed095b48184f706adf328
---

 libavcodec/agm.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libavcodec/agm.c b/libavcodec/agm.c
index c5c9a88dd9..bc9dfc02f3 100644
--- a/libavcodec/agm.c
+++ b/libavcodec/agm.c
@@ -1242,6 +1242,11 @@ static av_cold int decode_init(AVCodecContext *avctx)
     s->dct = avctx->codec_tag != MKTAG('A', 'G', 'M', '4') &&
              avctx->codec_tag != MKTAG('A', 'G', 'M', '5');
 
+    if (!s->rgb && !s->dct) {
+        if ((avctx->width & 1) || (avctx->height & 1))
+            return AVERROR_INVALIDDATA;
+    }
+
     avctx->idct_algo = FF_IDCT_SIMPLE;
     ff_idctdsp_init(&s->idsp, avctx);
     ff_init_scantable(s->idsp.idct_permutation, &s->scantable, ff_zigzag_direct);