[FFmpeg-devel] [PATCH 3/4] avcodec/ppc/fdctdsp: use more accurate constants

[Ganesh Ajjanagadde]

On Mon, Nov 16, 2015 at 1:20 PM, Hendrik Leppkes <h.leppkes at gmail.com> wrote:
> On Mon, Nov 16, 2015 at 6:24 PM, Ganesh Ajjanagadde <gajjanag at mit.edu> wrote:
>> On Mon, Nov 16, 2015 at 9:48 AM, Daniel Serpell <dserpell at gmail.com> wrote:
>>> Hi!,
>>>
>>> El Fri, Nov 13, 2015 at 11:42:30AM -0500, Ganesh Ajjanagadde escribio:
>>>> Whoever wrote this stuff had a pretty bad libm - digits differ pretty
>>>> quickly.
>>>
>>> They where correctly rounded to 24bit precision. I don't know if that
>>> was intentional, so I can't comment on the correctness of the patch.
>>
>> Unless I am off here, doing things at 24 bits makes very little sense:
>> float itself offers 32 bits
>
> Actually floats offer 23-bits of precision for the significand (with 8
> for mantissa and one sign bit) - 7-8 digits, so any more precision
> than that couldn't be stored accurately anymore, so when using single
> precision like this code, more precision is not needed.

Point taken. However, the general remark still applies: precision is
being lost unnecessarily at an intermediate computation stage since it
is anyway a static computation.

> Maybe it was even specifically wanted to have values that are
> represented accurately, but I do not know the history of this code
> either.
>
> - Hendrik
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel