[FFmpeg-devel] [PATCH] vp9: change type of tile_size from unsigned to int64_t

[Andreas Cadhalpun]

Otherwise the check 'tile_size < size' treats a negative size as
unsigned, causing the check to pass. This subsequently leads to
segmentation faults.

This was originally fixed as part of Libav commit 72ca83, so the
original author is one of the following developers:
        Anton Khirnov <anton at khirnov.net>
        Diego Biurrun <diego at biurrun.de>
        Luca Barbato <lu_zero at gentoo.org>
        Martin Storsjö <martin at martin.st>

Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun at googlemail.com>
---

Does someone still remember who authored this particular change,
so that he can get proper attribution?

---
 libavcodec/vp9.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libavcodec/vp9.c b/libavcodec/vp9.c
index c4efd42..d5147e5 100644
--- a/libavcodec/vp9.c
+++ b/libavcodec/vp9.c
@@ -4106,7 +4106,7 @@ static int vp9_decode_frame(AVCodecContext *ctx, void *frame,
                             tile_row, s->tiling.log2_tile_rows, s->sb_rows);
             if (s->pass != 2) {
                 for (tile_col = 0; tile_col < s->tiling.tile_cols; tile_col++) {
-                    unsigned tile_size;
+                    int64_t tile_size;
 
                     if (tile_col == s->tiling.tile_cols - 1 &&
                         tile_row == s->tiling.tile_rows - 1) {
-- 
2.1.4