[FFmpeg-cvslog] avcodec/hqx: Check the input data against the image size

[Michael Niedermayer]

ffmpeg | branch: master | Michael Niedermayer <michael at niedermayer.cc> | Sat Jul 20 23:37:38 2019 +0200| [07519f363bbd810e6ac8b187628e680e4e7026ca] | committer: Michael Niedermayer

avcodec/hqx: Check the input data against the image size

Fixes: Timeout (22 -> 100 ms)
Fixes: 15173/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HQX_fuzzer-5662556846292992
Fixes: 17896/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_HQX_fuzzer-5679312077848576

Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=07519f363bbd810e6ac8b187628e680e4e7026ca
---

 libavcodec/hqx.c | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/libavcodec/hqx.c b/libavcodec/hqx.c
index bc24ba91d1..39404d24e5 100644
--- a/libavcodec/hqx.c
+++ b/libavcodec/hqx.c
@@ -471,6 +471,14 @@ static int hqx_decode_frame(AVCodecContext *avctx, void *data,
     avctx->height              = ctx->height;
     avctx->bits_per_raw_sample = 10;
 
+    //The minimum size is 2bit per macroblock
+    // hqx_decode_422 & hqx_decode_444 have a unconditionally stored 4bits hqx_quants index
+    // hqx_decode_422a & hqx_decode_444a use cbp_vlc which has a minimum length of 2 bits for its VLCs
+    // The code rejects slices overlapping in their input data
+    if (avctx->coded_width / 16 * (avctx->coded_height / 16) *
+        (100 - avctx->discard_damaged_percentage) / 100 > 4LL * avpkt->size)
+        return AVERROR_INVALIDDATA;
+
     switch (ctx->format) {
     case HQX_422:
         avctx->pix_fmt = AV_PIX_FMT_YUV422P16;