[FFmpeg-devel] [PATCHv2] swscale: prevent undefined behaviour in the PUTRGBA macro

[Michael Niedermayer]

On Mon, Jul 08, 2024 at 12:25:17AM -0400, Sean McGovern wrote:
> ---
>  libswscale/yuv2rgb.c | 4 ++--
>  1 file changed, 2 insertions(+), 2 deletions(-)
> 
> diff --git a/libswscale/yuv2rgb.c b/libswscale/yuv2rgb.c
> index 977eb3a7dd..ac0b811f61 100644
> --- a/libswscale/yuv2rgb.c
> +++ b/libswscale/yuv2rgb.c
> @@ -100,9 +100,9 @@ const int *sws_getCoefficients(int colorspace)
>  
>  #define PUTRGBA(dst, ysrc, asrc, i, abase)                              \
>      Y              = ysrc[2 * i];                                       \
> -    dst[2 * i]     = r[Y] + g[Y] + b[Y] + (asrc[2 * i]     << abase);   \
> +    dst[2 * i]     = r[Y] + g[Y] + b[Y] + ((int32_t)(asrc[2 * i])     << abase);   \
>      Y              = ysrc[2 * i + 1];                                   \
> -    dst[2 * i + 1] = r[Y] + g[Y] + b[Y] + (asrc[2 * i + 1] << abase);
> +    dst[2 * i + 1] = r[Y] + g[Y] + b[Y] + ((int32_t)(asrc[2 * i + 1]) << abase);

can you explain what undefined behavior this does prevent and how ?
(in the commit message)

thx

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

The greatest way to live with honor in this world is to be what we pretend
to be. -- Socrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240709/53df255c/attachment.sig>