[FFmpeg-cvslog] swscale/utils: Limit filter shifting so as not to read from prior the array

[Michael Niedermayer]

ffmpeg | branch: release/1.1 | Michael Niedermayer <michaelni at gmx.at> | Thu Feb  5 00:12:08 2015 +0100| [097417299a0ef87cb3d459388a4b313272d74c38] | committer: Michael Niedermayer

swscale/utils: Limit filter shifting so as not to read from prior the array

Fixes out of array read
Fixes: asan_heap-oob_1fb2f9b_3780_cov_3984375136_usf.mkv

Found-by: Mateusz "j00ru" Jurczyk and Gynvael Coldwind
Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
(cherry picked from commit 692b22626ec9a9585f667c124a186b1a9796e432)

Signed-off-by: Michael Niedermayer <michaelni at gmx.at>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=097417299a0ef87cb3d459388a4b313272d74c38
---

 libswscale/utils.c |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/libswscale/utils.c b/libswscale/utils.c
index 55ade89..92e712b 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -571,14 +571,15 @@ static int initFilter(int16_t **outFilter, int32_t **filterPos,
         }
 
         if ((*filterPos)[i] + filterSize > srcW) {
-            int shift = (*filterPos)[i] + filterSize - srcW;
+            int shift = (*filterPos)[i] + FFMIN(filterSize - srcW, 0);
+
             // move filter coefficients right to compensate for filterPos
             for (j = filterSize - 2; j >= 0; j--) {
                 int right = FFMIN(j + shift, filterSize - 1);
                 filter[i * filterSize + right] += filter[i * filterSize + j];
                 filter[i * filterSize + j]      = 0;
             }
-            (*filterPos)[i]= srcW - filterSize;
+            (*filterPos)[i]-= shift;
         }
     }