[FFmpeg-devel] [PATCH] AAC decoder
Sat May 31 12:59:35 CEST 2008
On 29 May 2008, at 12:30, Michael Niedermayer wrote:
> On Thu, May 29, 2008 at 12:17:30PM +0100, Robert Swain wrote:
>> 2008/5/29 Robert Swain <robert.swain at gmail.com>:
>>> 2008/5/29 Robert Swain <robert.swain at gmail.com>:
>>>> I'm considering what precision to use for the tables. Using
>>>> %42.32f gives the first value as:
>>>> and the last as:
>>>> How does one decide such things? :)
>>> It seems scientific notation would be best for this. That should
>>> been obvious to me looking at those numbers. I've used 16 decimal
>>> places, is that OK? Looking at ivquant_tab now.
> The number of digits should be enough so that the float table is equal
> to what the generator produced. Iam not sure if this is true for
Tested manually by generating the tables with varying decimal places
and comparing to a float casted calculated result for each table value
and increasing the decimal places until all were the same.
>> Patch attached.
Applied with tested tables.
More information about the ffmpeg-devel