[FFmpeg-devel] [PATCH] avcodec/apedec: fix undefined left shifts of negative numbers

[Hendrik Leppkes]

On Sun, Sep 20, 2015 at 4:18 AM, Ganesh Ajjanagadde
<gajjanagadde at gmail.com> wrote:
> This fixes -Wshift-negative-value reported with clang 3.7+, e.g
> http://fate.ffmpeg.org/log.cgi?time=20150919172459&log=compile&slot=x86_64-darwin-clang-polly-notiling-3.7.
> Note that the patch crucially depends on int >= 32 bits,
> an assumption made in many places in the codebase.
>
> Signed-off-by: Ganesh Ajjanagadde <gajjanagadde at gmail.com>
> ---
>  libavcodec/apedec.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/libavcodec/apedec.c b/libavcodec/apedec.c
> index 5536e0f..7b34d26 100644
> --- a/libavcodec/apedec.c
> +++ b/libavcodec/apedec.c
> @@ -1281,7 +1281,7 @@ static void do_apply_filter(APEContext *ctx, int version, APEFilter *f,
>              /* Update the adaption coefficients */
>              absres = FFABS(res);
>              if (absres)
> -                *f->adaptcoeffs = ((res & (-1<<31)) ^ (-1<<30)) >>
> +                *f->adaptcoeffs = ((res & (-(1<<31))) ^ (-(1<<30))) >>
>                                    (25 + (absres <= f->avg*3) + (absres <= f->avg*4/3));
>              else
>                  *f->adaptcoeffs = 0;
> --
> 2.5.2
>

After this patch (GCC 5.2, x86)

libavcodec/apedec.c: In function 'do_apply_filter':
libavcodec/apedec.c:1284:44: warning: integer overflow in expression
[-Woverflow]
                 *f->adaptcoeffs = ((res & (-(1<<31))) ^ (-(1<<30))) >>