[FFmpeg-cvslog] avcodec/vp3: Add missing check for av_malloc

[Jiasheng Jiang]

ffmpeg | branch: release/2.8 | Jiasheng Jiang <jiasheng at iscas.ac.cn> | Tue Feb 15 17:58:08 2022 +0800| [e3bd8754ec4dbc5887fe1c02f15c775938509a86] | committer: Michael Niedermayer

avcodec/vp3: Add missing check for av_malloc

Since the av_malloc() may fail and return NULL pointer,
it is needed that the 's->edge_emu_buffer' should be checked
whether the new allocation is success.

Fixes: d14723861b ("VP3: fix decoding of videos with stride > 2048")
Reviewed-by: Peter Ross <pross at xvid.org>
Signed-off-by: Jiasheng Jiang <jiasheng at iscas.ac.cn>
(cherry picked from commit 656cb0450aeb73b25d7d26980af342b37ac4c568)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=e3bd8754ec4dbc5887fe1c02f15c775938509a86
---

 libavcodec/vp3.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/libavcodec/vp3.c b/libavcodec/vp3.c
index 00b005551f..2933544256 100644
--- a/libavcodec/vp3.c
+++ b/libavcodec/vp3.c
@@ -2095,8 +2095,13 @@ static int vp3_decode_frame(AVCodecContext *avctx,
     if (ff_thread_get_buffer(avctx, &s->current_frame, AV_GET_BUFFER_FLAG_REF) < 0)
         goto error;
 
-    if (!s->edge_emu_buffer)
+    if (!s->edge_emu_buffer) {
         s->edge_emu_buffer = av_malloc(9 * FFABS(s->current_frame.f->linesize[0]));
+        if (!s->edge_emu_buffer) {
+            ret = AVERROR(ENOMEM);
+            goto error;
+        }
+    }
 
     if (s->keyframe) {
         if (!s->theora) {