[FFmpeg-cvslog] avcodec/apedec: more checks for k

[Michael Niedermayer]

ffmpeg | branch: release/2.1 | Michael Niedermayer <michaelni at gmx.at> | Mon Dec 23 18:09:58 2013 +0100| [5f56e495aea44bd43d24e237e898b8a8aa6726e6] | committer: Michael Niedermayer

avcodec/apedec: more checks for k

Fixes assertion failure
Fixes part of msan_uninit-mem_7fa0d8c8bd58_8417_sh3.ape
Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit d5128fce38646d3f64c55feda42084888ba0e87e)

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=5f56e495aea44bd43d24e237e898b8a8aa6726e6
---

 libavcodec/apedec.c |    4 ++++
 1 file changed, 4 insertions(+)

diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
index ea71ea0..325f36d 100644
--- a/libavcodec/apedec.c
+++ b/libavcodec/apedec.c
@@ -603,10 +603,14 @@ static void decode_array_0000(APEContext *ctx, GetBitContext *gb,
         rice->ksum += out[i];
     }
     rice->k = av_log2(rice->ksum / 10) + 1;
+    if (rice->k >= 24)
+        return;
     for (; i < 64; i++) {
         out[i] = get_rice_ook(&ctx->gb, rice->k);
         rice->ksum += out[i];
         rice->k = av_log2(rice->ksum / ((i + 1) * 2)) + 1;
+        if (rice->k >= 24)
+            return;
     }
     ksummax = 1 << rice->k + 7;
     ksummin = rice->k ? (1 << rice->k + 6) : 0;