[FFmpeg-devel] [PATCH] checkasm/af_afir: relax the max allowed absolute difference

[Hendrik Leppkes]

On Fri, Jan 11, 2019 at 2:12 PM Derek Buitenhuis
<derek.buitenhuis at gmail.com> wrote:
>
> On 10/01/2019 23:34, James Almer wrote:
> > -        if (!float_near_abs_eps(cdst[i], odst[i], FLT_EPSILON)) {
> > +        if (!float_near_abs_eps(cdst[i], odst[i], 6.2e-05)) {
>
> Can you elaborate a bit more on this? FLT_EPSILON is used correctly here
> as far as I can tell, and it is not clear why it fails on x86_32, and why
> we should choose an arbitrary unportable number instead (who knows if it
> explodes on weird systems).
>

Because the computation accumulates more inaccuarcy then FLT_EPSILON
allows for. That value is really not of that great use. If you have
two accurate numbers and do one calculation, it may work, but if you
do a whole bunch of them, the error accumulates and eventually gets
bigger then FLT_EPSILON.
x86_32 floating point is for $reasons a tad bit less accurate then on
x86_64, for example, resulting in the test failing. We have some other
float tests that  do (or used to) fail sporadically due to inaccuracy
problems, which sometimes where fixed by similar means - or
multiplifying FLT_EPSILON to make it bigger.

- Hendrik