[FFmpeg-devel] [PATCH 2/5] lavfi/gradfun: fix rounding in MMX code.

[Loren Merritt]

On Tue, 18 Dec 2012, Clément Bsch wrote:
> On Tue, 18 Dec 2012, Clément Bsch wrote:
>> On Fri, Dec 07, 2012 at 11:49:40PM +0000, Loren Merritt wrote:
>>
>>> The original order probably was about avoiding overflow, but limiting
>>> strength is better.
>>
>> Just to make sure I understand the issue correctly:
>>
>> In the ASM, delta is expressed as u16, so delta<<2 will overflow for
>> values > 0x3fff. In these cases we want that m, defined by:
>>     int m = abs(delta) * thresh >> 16;     (1)
>>     m = FFMAX(0, 127 - m);
>> ...to be 0, so whatever the value of delta, the expression:
>>     m = m * m * delta >> 14;
>> ...will always be 0.
>>
>> To do so, we just need to make sure that abs(delta) * thresh >> 16 to be
>> always >= 127.
>>
>> We currently have the threshold in the range from (1<<15)/0.51 to
>> (1<<15)/255, with [0.51;255] being the range the user can specify. The
>> higher the user specified threshold, the closer we get to 127 (from +oo).
>>
>> We need to solve:
>>
>>    delta * ((1<<15)/x) >> 16 >= 127 (based on (1)),
>>      with delta > 0x3fff and x >= 0.51
>>
>> => delta * (1<<15)/x / (1<<16) >= 127
>>    delta / (2x) >= 127
>>
>> so we have 0.51 <= x <= 2 * delta / 127
>
> Here is where is my mistake, the solution is 0.51 <= x <= 129/2, which is
> indeed not in the range.
>
> Does a threshold > 64 makes sense?

No. A threshold of 64 makes gradfun pretty much equivalent to blur. I have
never seen a video that wanted >5.

--Loren Merritt