[FFmpeg-cvslog] avcodec/omx: Do not pass negative value into av_malloc()

[Michael Niedermayer]

ffmpeg | branch: release/3.2 | Michael Niedermayer <michael at niedermayer.cc> | Thu Dec 29 02:19:27 2016 +0100| [7d222736c27ef7aa0880fc91a51d83dfeac7b388] | committer: Michael Niedermayer

avcodec/omx: Do not pass negative value into av_malloc()

Fixes CID1396849

Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>
(cherry picked from commit bd83c295fc1b7f8001e5d134b912af86cd62c3f2)
Signed-off-by: Michael Niedermayer <michael at niedermayer.cc>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=7d222736c27ef7aa0880fc91a51d83dfeac7b388
---

 libavcodec/omx.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/libavcodec/omx.c b/libavcodec/omx.c
index c1b6fb9..2fe2142 100644
--- a/libavcodec/omx.c
+++ b/libavcodec/omx.c
@@ -761,7 +761,10 @@ static int omx_encode_frame(AVCodecContext *avctx, AVPacket *pkt,
             } else {
                 // If not, we need to allocate a new buffer with the right
                 // size and copy the input frame into it.
-                uint8_t *buf = av_malloc(av_image_get_buffer_size(avctx->pix_fmt, s->stride, s->plane_size, 1));
+                uint8_t *buf = NULL;
+                int image_buffer_size = av_image_get_buffer_size(avctx->pix_fmt, s->stride, s->plane_size, 1);
+                if (image_buffer_size >= 0)
+                    buf = av_malloc(image_buffer_size);
                 if (!buf) {
                     // Return the buffer to the queue so it's not lost
                     append_buffer(&s->input_mutex, &s->input_cond, &s->num_free_in_buffers, s->free_in_buffers, buffer);