[FFmpeg-devel] [PATCH] swscale/utils: disable full_chr_h_input optimization for odd width

[Niklas Haas]

From: Niklas Haas <git at haasn.dev>

The basic problem here is that the rgb*ToUV_half_* functions hard-code a
bilinear downsample from src[i] + src[i+1], with no bounds check on the i+1
access.

Due to the signature of the function, we cannot easily plumb the "true" width
into the function body to perform a bounds check. Similarly, we cannot easily
pre-pad the input because it is typically reading from the original input
frame, which would require a full memcpy to pad. Either of these solutions are
more trouble than the feature is worth, so just disable it on odd input sizes.

Fixes: use of uninitialized value
Fixes: ticket #11265
Signed-off-by: Niklas Haas <git at haasn.dev>
Sponsored-by: Sovereign Tech Fund
---
 libswscale/utils.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/libswscale/utils.c b/libswscale/utils.c
index 26ea6062a0..32f90e366e 100644
--- a/libswscale/utils.c
+++ b/libswscale/utils.c
@@ -1535,7 +1535,7 @@ av_cold int ff_sws_init_single_context(SwsContext *sws, SwsFilter *srcFilter,
 
     /* drop every other pixel for chroma calculation unless user
      * wants full chroma */
-    if (isAnyRGB(srcFormat) && !(flags & SWS_FULL_CHR_H_INP)   &&
+    if (isAnyRGB(srcFormat) && !(srcW & 1) && !(flags & SWS_FULL_CHR_H_INP) &&
         srcFormat != AV_PIX_FMT_RGB8 && srcFormat != AV_PIX_FMT_BGR8 &&
         srcFormat != AV_PIX_FMT_RGB4 && srcFormat != AV_PIX_FMT_BGR4 &&
         srcFormat != AV_PIX_FMT_RGB4_BYTE && srcFormat != AV_PIX_FMT_BGR4_BYTE &&
-- 
2.47.0