[FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

[James Almer]

On 1/11/2019 12:17 PM, Derek Buitenhuis wrote:
> On 11/01/2019 13:28, Hendrik Leppkes wrote:
>> Because the computation accumulates more inaccuarcy then FLT_EPSILON
>> allows for. That value is really not of that great use. If you have
>> two accurate numbers and do one calculation, it may work, but if you
>> do a whole bunch of them, the error accumulates and eventually gets
>> bigger then FLT_EPSILON.
>> x86_32 floating point is for $reasons a tad bit less accurate then on
>> x86_64, for example, resulting in the test failing. We have some other
>> float tests that  do (or used to) fail sporadically due to inaccuracy
>> problems, which sometimes where fixed by similar means - or
>> multiplifying FLT_EPSILON to make it bigger.
> 
> OK.
> 
> Two things:
> 
>     1. That should be in the commit messages.
>     2. Would some multiple of FLT_EPSILON make more sense?

Michael suggested 1000*FLT_EPSILON but IMO that's too big and may hide
errors in future implementations.
The value i used is the smallest value i found that didn't fail after
several runs. 6.1e-05 for example fails.

> 
> - Derek
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
>