[FFmpeg-devel] Audio conversion and floating-point codecs
Sat May 15 21:17:51 CEST 2010
There is a long-standing desire from some to make the floating-point
decoders output float samples instead of converting to int16
internally, and I agree with the reasons for this. However, making
this change hastily will make decoding orders of magnitude slower on
many CPUs. The reason is that when a decoder outputs float samples,
the fast asm code for float-to-int conversion is not used.
In order to change the output format of these decoders without
impacting performance, we must first make a few improvements to the
avcodec API and to the generic audio format conversion code.
What we have
- Very fast float-to-int16 conversion code in dsputil. These
functions require input scaled to -32k..32k.
- The codecs in question all scale the output to the correct range as
part of transforms or filters. The scaling is thus effectively free.
- Generic sample format conversion code (audioconvert.c). This code
requires float input in the range -1..1. It does not use any asm
and is thus excruciatingly slow. Decoding wmapro on Cortex-A8
spends more than 50% of the total time here.
What we need
- The libavcodec API needs to be amended such that a specific scaling
can be requested of the decoders. This should probably be done
similarly to how channel down-mixing is already handled.
- The decoders should output planar audio instead of interleaved for
multichannel streams. This probably means introducing
avcodec_decode_audio4() with an AVFrame output.
- A better audio conversion system needs to be implemented. Ideally,
this should be able to reuse existing asm code. To this end, the
desired input range should be exported after configuration, allowing
it to be passed back to decoders.
- The conversion system should be designed to allow up/down-mixing of
channels within the same API. This is a feature currently missing
from FFmpeg, making playback of multi-channel AAC or Vorbis on a
stereo output difficult. Implementing this is not a prerequisite
for switching the output format of the decoders.
- All the pieces above need to be tied together in ffmpeg.c.
None of the above is especially difficult to do, but it is important
that it is done properly, or performance will suffer.
mans at mansr.com
More information about the ffmpeg-devel