[FFmpeg-cvslog] lsws: prevent overflow in sws_init_context()

[Stefano Sabatini]

ffmpeg | branch: oldabi | Stefano Sabatini <stefano.sabatini-lala at poste.it> | Mon Apr 25 01:17:08 2011 +0200| [bd2a3700c045201b043a0e812d932e9d4fc37e82] | committer: Stefano Sabatini

lsws: prevent overflow in sws_init_context()

In the loop:
    for (i=0; i<dstH; i++) {
        int chrI= i*c->chrDstH / dstH;

when i*c->chrDstH > INT_MAX this leads to an integer overflow, which
results in a negative value for chrI and in out-of-buffer reads. The
overflow is avoided by forcing int64_t arithmetic by casting i to
int64_t.

Fix crash, and trac issue #72.

Signed-off-by: Stefano Sabatini <stefano.sabatini-lala at poste.it>

> http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=bd2a3700c045201b043a0e812d932e9d4fc37e82
---

 libswscale/utils.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/libswscale/utils.c b/libswscale/utils.c
index 6e8e40b..1f4a6c4 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -1000,7 +1000,7 @@ int sws_init_context(SwsContext *c, SwsFilter *srcFilter, SwsFilter *dstFilter)
     c->vLumBufSize= c->vLumFilterSize;
     c->vChrBufSize= c->vChrFilterSize;
     for (i=0; i<dstH; i++) {
-        int chrI= i*c->chrDstH / dstH;
+        int chrI= (int64_t)i*c->chrDstH / dstH;
         int nextSlice= FFMAX(c->vLumFilterPos[i   ] + c->vLumFilterSize - 1,
                            ((c->vChrFilterPos[chrI] + c->vChrFilterSize - 1)<<c->chrSrcVSubSample));