[FFmpeg-cvslog] avcodec/qdm2: Check frame size

[Michael Niedermayer]

ffmpeg | branch: release/4.2 | Michael Niedermayer <michael at niedermayer.cc> | Thu Aug 29 21:26:43 2019 +0200| [e66d4725c7187d291edc9b1076585b1a48da907e] | committer: Michael Niedermayer

avcodec/qdm2: Check frame size

Fixes: index 2304 out of bounds for type 'float [2304]'
Fixes: 16332/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_QDM2_fuzzer-5679142481166336

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit 12b909ba319d32ed100d9b26021aa9b6976424d7)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e66d4725c7187d291edc9b1076585b1a48da907e
---

 libavcodec/qdm2.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/libavcodec/qdm2.c b/libavcodec/qdm2.c
index eaffb36dcc..ac8ae8cbbb 100644
--- a/libavcodec/qdm2.c
+++ b/libavcodec/qdm2.c
@@ -1727,6 +1727,11 @@ static av_cold int qdm2_decode_init(AVCodecContext *avctx)
     s->sub_sampling = s->fft_order - 7;
     s->frequency_range = 255 / (1 << (2 - s->sub_sampling));
 
+    if (s->frame_size * 4 >> s->sub_sampling > MPA_FRAME_SIZE) {
+        avpriv_request_sample(avctx, "large frames");
+        return AVERROR_PATCHWELCOME;
+    }
+
     switch ((s->sub_sampling * 2 + s->channels - 1)) {
         case 0: tmp = 40; break;
         case 1: tmp = 48; break;