[FFmpeg-devel] [RFC] libavfilter audio - af_resample

[S.N. Hemanth Meenakshisundaram]

Michael Niedermayer wrote:
> On Sat, Jul 10, 2010 at 11:37:54PM +0100, M?ns Rullg?rd wrote:
>   
>> Michael Niedermayer <michaelni at gmx.at> writes:
>>
>>     
>>> On Sat, Jul 10, 2010 at 12:05:43PM +0100, M?ns Rullg?rd wrote:
>>>       
>>>> "S.N. Hemanth Meenakshisundaram" <smeenaks at ucsd.edu> writes:
>>>>
>>>>         
>>>>> 2. Earlier, the channel conversion functions only worked for short
>>>>> (SAMPLE_FMT_S16) pointers but writing a different version for each
>>>>> will increase the number of functions required by 5x. To get around
>>>>> this, I have chosen to always convert incoming audio data to
>>>>> SAMPLE_FMT_S16. Most of the times, this conversion is not necessary
>>>>> since most codecs output S16 data. Once we ensure data is in S16
>>>>> format, channel downmix/upmix is done and finally this data is
>>>>> converted to required output sample format (in case it is not
>>>>> SAMPLE_FMT_S16).
>>>>>           
>>>> [...]
>>> i would suggest we support s16->s16 and float->float mixers and than a
>>> full set of X->Y sample format converters to connect to them
>>> sample formats different from s16 and float are rater rare so this should
>>> be ok
>>>       
>> The interface still has to accept any format for input and output.
>> Converting everything to s16 or float internally, to favour speed or
>> accuracy, is probably acceptable.  It should nevertheless be designed
>> such that specific conversions can be easily added later.
>>     
>
> of course
>   
Hi All,

A few clarifications and some questions:

The interface does accept any sample format and output sample format too 
can be any of the 5. The S16 conversion is internal and only for the S16 
mixers (where mixing is required). I will add float mixers if needed.

>
>   
>>>> I dislike the idea of having basic mixing functionality only available
>>>> as a full-blown filter.
>>>>         
>>> i do like having just a single api through which any mix+sample convertion
>>> can be requested and then the best and fastest implementation for this
>>> is automatically selected
>>>
>>> The reason why i like this, is that a single simple API is easy to document
>>> remember and use. Also its very easy to export the subparts individually
>>> at a later point if there is anyone wanting that.
>>>       
>> [...]
>> Being able to do a simple conversion
>> without invoking all the might of libavfilter is desirable.
>>     
> [...]
>   

I understand that it would be clumsy to call af_resample and lavfi every 
time a sample format conversion or channel mixing is required 
internally. However, wouldn't af_resample still be useful by playing the 
same role as vf_scale for video.

If there are two random audio filters that only operate on s16 and float 
data respectively, then a af_resample filter could be inserted in the 
lavfi chain to do the required conversion (and nothing more).

Of course, vf_scale internally calls libswscale while af_resample is 
currently duplicating a lot of lavc's resample & audioconvert.c code 
internally so I could make changes if necessary.

Should I instead wrap audioconvert for sample format conversion (use 
av_audio_convert) and wrap resample.c for channel mixing. Also, perhaps 
af_resample should be renamed to reformat to avoid confusing with the 
downsampling/upsampling in lavc resample2.c

Please let me know.

Regards,
Hemanth