[FFmpeg-devel] [RFC] swscale dithering

[Michael Niedermayer]

Hi Niklas

On Mon, Mar 24, 2025 at 01:43:19PM +0100, Niklas Haas wrote:
> Hi all,
> 
> As part of my ongoing swscale rewrite, we have both the opportunity and the
> need to make a central decision about how to apply rounding and/or dithering.
> 
> Some particular cases I want to point out and gather feedback on include:
> 

all IMHO:


> 1. Should we dither and/or accurately round when scaling up full range
>    content? For example, say you are converting from full-range rgb24 to

by default, yes


[...]

> 2. At what bit depth does dithering become negligible? For context, the

I think we should consistently always apply it by default

also especially if someone does work with lets say 32bit, that person has
some strange requirements already and direct human vison may not be it.


[...]

> 3. Should we dither per-channel after conversion from grayscale to RGB? For

in general, yes


[...]
> 
> 3. What should we make of the SWS_ACCURATE_RND and SWS_BITEXACT flags? I am
>    personally thinking that SWS_BITEXACT should become a no-op flag, with
>    bit exact output being the default behavior of all new implementations.
>    But What about SWS_ACCURATE_RND?
> 
>    I am thinking that SWS_ACCURATE_RND should essentially be the switch that
>    toggles our preferred resolution of question 1. So in other words, with
>    SWS_ACCURATE_RND specified, full range upconversions should go through an
>    accurate dither pass, while being relaxed to the simple (x << 2) | (x >> 6)
>    upconversion in the absence of this flag.
> 
>    How should this flag relate to question 2? With the flag specified, I am
>    thinking that we should also force dithering even at 16 bit depth, and
>    skip dithering in this case only in the flag's absence. If so, what
>    bit depth should the cutoff threshold be, for when to skip accurate
>    dithering? I am thinking to simply use the 12/14 bit SDR/HDR threshold as
>    appropriate for the content type.
> 
> This would lead to the following conversions, as an illustration:
> 
> SWS_ACCURATE_RND specified:
> 
> - rgb24 -> yuv420p10:       full dithering
> - rgb24 -> yuv420p12:       full dithering
> - rgb24 -> rgb30:           full dithering
> - rgb24 -> rgba64:          full dithering
> - yuva444p -> yuva444p10:   scale YUV, dither alpha
> - yuva444p14 -> yuva444p16: scale YUV, dither alpha
> - yuv444p10 -> yuv444p14:   left shift, no dithering needed
> 
> SWS_ACCURATE_RND absent:
> 
> - rgb24 -> yuv420p10:       full dithering
> - rgb24 -> yuv420p12:       truncate if SDR, full dithering if HDR
> - rgb24 -> rgb30:           truncate
> - rgb24 -> rgba64:          truncate
> - yuva444p -> yuva444p10:   left shift YUV, truncate alpha
> - yuva444p14 -> yuva444p16: left shift YUV, truncate alpha
> 
> Does this seem reasonable?

IMHO in the accurate mode, dither should always be on
its also easier to understand

but there could be a flag for vissual percetion

SWS_VISSUAL_PERCEPTION (or some better name)
some flag that uses less accurate and faster operations when their
effect is expected to be vissually impercivable
(this may be percievable when contrast, color, resolution or other
 is changed)

thx

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have often repented speaking, but never of holding my tongue.
-- Xenocrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20250324/790b041e/attachment.sig>