[FFmpeg-devel] [PATCH] swresample/resample: do not allow negative dst_size return value

[Michael Niedermayer]

On Tue, Dec 13, 2016 at 02:16:03PM +0700, Muhammad Faiz wrote:
> This should fix Ticket6012
> 
> Signed-off-by: Muhammad Faiz <mfcc64 at gmail.com>
> ---
>  libswresample/resample.c | 7 ++++---
>  1 file changed, 4 insertions(+), 3 deletions(-)
> 
> diff --git a/libswresample/resample.c b/libswresample/resample.c
> index 71dffb9..ce6a82f 100644
> --- a/libswresample/resample.c
> +++ b/libswresample/resample.c
> @@ -478,8 +478,9 @@ static int swri_resample(ResampleContext *c,
>          int64_t incr= (1LL<<32) * c->dst_incr / c->src_incr;
>          int new_size = (src_size * (int64_t)c->src_incr - frac + c->dst_incr - 1) / c->dst_incr;
>  
> -        dst_size= FFMIN(dst_size, new_size);
> -        c->dsp.resample_one(dst, src, dst_size, index2, incr);
> +        dst_size = FFMAX(FFMIN(dst_size, new_size), 0);
> +        if (dst_size > 0)
> +            c->dsp.resample_one(dst, src, dst_size, index2, incr);
>  
>          index += dst_size * c->dst_incr_div;
>          index += (frac + dst_size * (int64_t)c->dst_incr_mod) / c->src_incr;
> @@ -494,7 +495,7 @@ static int swri_resample(ResampleContext *c,
>          int64_t delta_frac = (end_index - c->index) * c->src_incr - c->frac;
>          int delta_n = (delta_frac + c->dst_incr - 1) / c->dst_incr;
>  
> -        dst_size = FFMIN(dst_size, delta_n);
> +        dst_size = FFMAX(FFMIN(dst_size, delta_n), 0);

Shouldnt this and other things be done outside the loop that
calls swri_resample() ?

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The real ebay dictionary, page 1
"Used only once"    - "Some unspecified defect prevented a second use"
"In good condition" - "Can be repaird by experienced expert"
"As is" - "You wouldnt want it even if you were payed for it, if you knew ..."
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20161213/839b8636/attachment.sig>