[FFmpeg-devel] [RFC] libavfilter audio API and related issues

[S.N. Hemanth Meenakshisundaram]

On 07/01/2010 03:14 PM, Stefano Sabatini wrote:
> On date Friday 2010-06-25 03:52:45 -0700, S.N. Hemanth Meenakshisundaram encoded:
>    
>> Hi All,
>> [...] Is it ok to name this filter af_resample or should I
>> name this one af_reformat and reserve the af_resample name for the
>> sample rate conversion filter?
>>      
> Yes I prefer the second scheme:
> af_reformat
> af_resample
>
>    

Hi Stefano,

I'll rename this as suggested.

> is consistent with the video scale / format filters scheme, maybe even
> af_format ->  aformat will do.
>
>    
>> 2. The current resample.c only accepts two channel input data - why?
>> Depending on the file and codec, isn't it possible for input data to
>> be more than two channels? Can I attempt adding multichannel input
>> support or is there a reason it is not supported?
>>      
> Audio multichannel support is welcome, and I believe is not
> implemented for historical reasons (the same for which most codecs
> assume the sample format is always signed 16 bit).
>    

Did you also get a chance to see my patch from yesterday. The filter now 
works for the existing channel conversions (if output sample format is 
16-bit) and for any sample format conversion. I am now writing a channel 
conversion routine for downmixing 5.1 audio.

However, the SDL_AudioOpen call configures the audio output to 16-bit 
signed and stereo output during init. So even though the filter works, I 
am not able to get SDL to play the audio right unless I set the SDL 
parameters to the new values in the code and build again. How do I 
reconfigure SDL whenever channel/sample format changes. Is closing and 
reopening SDL session acceptable?


>    
>> 3. The current output sample format is always forced to S16. The
>> channel layout conversion filters after this all assume that the
>> individual samples are shorts. This needs to be fixed right? For
>> example, I was thinking of changing stereo_to_mono function so it
>> accepts an extra stride parameter and uses that to do its job
>> irrespective of whether inputs are shorts (S16) or otherwise. Is
>> this ok?
>>      
> Yes and yes, and I would like to hear someone which worked extensively
> with the audio API to say which are the potential problems/warts to
> keep in mind for such an implementation.
>    

I have been trying to think of a neat way of removing the 'short' 
assumption in channel conversion functions. But either branching in the 
regular audio path or writing multiple versions of the channel 
conversion functions for each sample format seems to be necessary. I 
will post a patch based on the first approach shortly.

Again, SDL only seems to support 8-bit and 16-bit integer sample 
formats. Other sample formats would still be useful for ffmpeg I guess.

> Resuming:
> * audioconvert.h/audioconvert.c:
>      struct AVAudioConvert
>      generic sample and channel layout API
>      av_audio_convert* API, supports up to 6 channels
>
> * avcodec.h/resample.c
>      struct ReSampleContext
>      audio_resample_init() - deprecated
>      av_audio_resample_init()
>      audio_resample()
>      audio_resample_close()
>
> * avcodec.h/resample2.c
>      struct AVResampleContext
>      av_resample_init()
>      av_resample_close()
>      av_resample_compensate()
>      av_resample()
>
> Can someone say which is the main difference between the various APIs
> and what's the plan for their support (i.e. which is going to be
> deprecated in the future?).
>
> This API seems to need anyway some (major?) rework, I already proposed
> some time ago to create a separate lib for this (libavresample -
> lavre?), which would have a role similar to that of libswscale.
>
>    
>> I still need to fix nits etc in af_resample. Will do those when
>> finishing the channel layout conversion.
>>      
> For the moment I believe that we need to focus on getting the first
> simple version of the audio framework in place, then we will hopefully
> work through the audio resampling framework current limitations.
>
> Regards.
>    

Now that the framework and resample (reformat) filter both work, I will 
add 5.1 downmix, multiple sample format support during channel 
conversion and later ffmpeg support.

Regards,
Hemanth