[FFmpeg-devel] [FFmpeg-cvslog] swscale: use 16-bit intermediate precision for RGB/XYZ conversion

[Niklas Haas]

On Fri, 14 Mar 2025 00:54:25 +0100 Michael Niedermayer <michael at niedermayer.cc> wrote:
> Hi
> 
> On Sun, Jan 26, 2025 at 07:26:59PM +0100, Michael Niedermayer wrote:
> > On Thu, Dec 26, 2024 at 07:33:23PM +0000, Niklas Haas wrote:
> > > ffmpeg | branch: master | Niklas Haas <git at haasn.dev> | Mon Dec 16 14:49:39 2024 +0100| [af6d52eec66961f6a502b0f2f390c12226d087cd] | committer: Niklas Haas
> > > 
> > > swscale: use 16-bit intermediate precision for RGB/XYZ conversion
> > > 
> > > The current logic uses 12-bit linear light math, which is woefully insufficient
> > > and leads to nasty postarization artifacts. This patch simply switches the
> > > internal logic to 16-bit precision.
> > > 
> > > This raises the memory requirement of these tables from 32 kB to 272 kB.
> > > 
> > > All relevant FATE tests updated for improved accuracy.
> > > 
> > > Fixes: #4829
> > > Signed-off-by: Niklas Haas <git at haasn.dev>
> > > Sponsored-by: Sovereign Tech Fund
> > > 
> > > > http://git.videolan.org/gitweb.cgi/ffmpeg.git/?a=commit;h=af6d52eec66961f6a502b0f2f390c12226d087cd
> > > ---
> > 
> > this breaks on x86-32
> 
> ping
> 
> the fate tests for a major architecture are broken since this
> This impacts future bisections over that range for example

My bad, missed this email for some reason.

Will investigate and fix tomorrow.

> 
> you can trivially test this on x86-64 with something like:
> --cross-prefix=/usr/i686-linux-gnu/bin/ --cc='ccache i686-linux-gnu-gcc-7' --arch=x86_32  --target-os=linux --enable-cross-compile
> if you have the correponding packages installed
> 
> before af6d52eec66961f6a502b0f2f390c12226d087cd
> TEST    filter-pixdesc-xyz12be
> 
> after af6d52eec66961f6a502b0f2f390c12226d087cd:
> --- src/tests/ref/fate/filter-pixdesc-xyz12be	2025-03-14 00:46:40.445681223 +0100
> +++ tests/data/fate/filter-pixdesc-xyz12be	2025-03-14 00:47:08.301903796 +0100
> @@ -1 +1 @@
> -pixdesc-xyz12be     1508a33dea936c45d9ee13f7743af00d
> +pixdesc-xyz12be     198f43f452bc55f4ca1e0e0171de5c4c
> Test filter-pixdesc-xyz12be failed. Look at tests/data/fate/filter-pixdesc-xyz12be.err for details.
> make: *** [src/tests/Makefile:311: fate-filter-pixdesc-xyz12be] Error 1
> 
> in master:
> --- src/tests/ref/fate/filter-pixdesc-xyz12be	2025-03-14 00:46:40.445681223 +0100
> +++ tests/data/fate/filter-pixdesc-xyz12be	2025-03-14 00:48:43.042660972 +0100
> @@ -1 +1 @@
> -pixdesc-xyz12be     1508a33dea936c45d9ee13f7743af00d
> +pixdesc-xyz12be     198f43f452bc55f4ca1e0e0171de5c4c
> Test filter-pixdesc-xyz12be failed. Look at tests/data/fate/filter-pixdesc-xyz12be.err for details.
> make: *** [src/tests/Makefile:315: fate-filter-pixdesc-xyz12be] Error 1
> 
> 
> thx
> 
> [...]
> 
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Never trust a computer, one day, it may think you are the virus. -- Compn