[FFmpeg-devel] [PATCH] Use larger tables for yuv > 8 bit to RGB conversion.

[Reimar Döffinger]

On 09.11.2013, at 23:43, Michael Niedermayer <michaelni at gmx.at> wrote:
> On Sat, Nov 09, 2013 at 08:59:46PM +0100, Reimar Döffinger wrote:
>> This should allow for fairly precise YUV16 to RGB48 conversion
>> for example.
> 
>> However I believe that this specific implementation is not as accurate
>> as it could/should be, i.e. the table generation might be buggy.
> 
> yes, seems so
> -vf format=yuv420p12,scale,format=rgb4_byte
> -vf format=gbrp12le,scale=flags=1,format=rgb4_byte
> breaks totally for example

I believe that is not the issue I meant. That should be a case where I did not update the output.c code correctly.
What I am more worried about is subtle rounding errors that cause the 10 bit tables to be maybe even less accurate than the 8 bit ones.

>> In addition it make the scaled yuv->rgb slightly slower, though also
>> more precise for 9 and 10 bit.
> 
>> Still not sure it is a good idea.
> 
> neither am i, but it seems more buggy than the shift
> implementation ATM

Don't worry about the bugs, that "just" needs more time, reviews and probably some directed FATE tests.
I just didn't want to put the time into it if we'd say "forget it, just do the shift approach".
> 
Though I am starting to think that maybe more than 8 bit precision support in the C yuv to rgb code would be good and worth a bit of effort and minor slowdown (the scaled pure C code is really slow anyway...)