[FFmpeg-devel] [PATCH] avcodec/h264_mb: Fix undefined shifts

[Ronald S. Bultje]

Hi,

On Wed, Mar 11, 2015 at 9:00 PM, Michael Niedermayer <michaelni at gmx.at>
wrote:

> Found-by: Clang -fsanitize=shift
> Reported-by: Thierry Foucu <tfoucu at google.com>
> Signed-off-by: Michael Niedermayer <michaelni at gmx.at>
> ---
>  libavcodec/h264_mb.c |   14 +++++++-------
>  1 file changed, 7 insertions(+), 7 deletions(-)
>
> diff --git a/libavcodec/h264_mb.c b/libavcodec/h264_mb.c
> index dd406c7..a4653aa 100644
> --- a/libavcodec/h264_mb.c
> +++ b/libavcodec/h264_mb.c
> @@ -213,7 +213,7 @@ static av_always_inline void mc_dir_part(H264Context
> *h, H264Picture *pic,
>      const int mx      = h->mv_cache[list][scan8[n]][0] + src_x_offset * 8;
>      int my            = h->mv_cache[list][scan8[n]][1] + src_y_offset * 8;
>      const int luma_xy = (mx & 3) + ((my & 3) << 2);
> -    ptrdiff_t offset  = ((mx >> 2) << pixel_shift) + (my >> 2) *
> h->mb_linesize;
> +    ptrdiff_t offset  = (mx >> 2) * (1 << pixel_shift) + (my >> 2) *
> h->mb_linesize;


Why is this undefined?

Ronald