[Ffmpeg-devel] channel ordering and downmixing

Justin Ruggles justinruggles
Sun Apr 8 20:56:46 CEST 2007


Michael Niedermayer wrote:
> Hi
> 
> On Sun, Apr 08, 2007 at 01:15:21AM -0400, Justin Ruggles wrote:
> 
>>Michael Niedermayer wrote:
>>
>>>i even suggest to completely ignore downmix coeffs first and design just the
>>>channel layout stuff that way we have a smaller set of things to consider
>>>and after the channel layout is done and in svn work on the downmix coeffs
>>>(just a random suggestion)
>>
>>Here is another concept-patch for channel layout handling.  Downmixing
>>is ignored for now.
>>
>>Demuxing:
>>The demuxer sets AVCodecContext->channel_layout to the appropriate
>>layout or leaves it NULL if unknown or codec-dependent.
>>
>>Decoding:
>>If required, the decoder sets AVCodecContext->channel_layout to the
>>channel layout of the input audio stream.  If downmixing is done in the
>>decoder (current situation), the final downmixed channel layout is
>>considered the input layout.
>>
>>Encoding:
>>The user must set AVCodecContext->channel_layout to one of the encoder's
>>compatible layouts in AVCodec->channel_layouts.  If the encoder's list
>>is empty, any layout can be used.
>>
>>Muxing:
>>The user's chosen layout must also match one of the muxer's compatible
>>layouts in AVOutputFormat->channel_layouts.
>>
>>Channel mixing and reordering API:
>>
>>av_channel_mix_init allocates the AVChannelMixContext and sets the
>>values from input parameters.  To help the user with finding compatible
>>layouts, this function returns NULL if the ilayout and olayout
>>parameters are not compatible.
> 
> 
> how can they be incompatible?

Well, without downmixing, different numbers of channels makes them
incompatible.  But even with the same number of channels, if the actual
channel locations contained in the layouts are different, there is no
way for the system to know which channel locations to map to eachother.
 For example, { L, R, Ls, Rs } vs. { C, L, R, Cs }.  Now if we ever get
to the point where there is really smart mixing, it could know to mix
L/R into C and mix Ls/Rs into Cs, then reorder as needed.  But when we
have that functionality, the 2 layouts could be considered compatible.

I guess if it ever gets good enough, any channel combination could be
mixed into any other combination by trying to recreate the sound field
as best it can based on speaker locations.  But until then...we need a
way to avoid combinations we can't handle yet.

I have done some thinking about how this is handled though.  In the last
patch, I proposed returning NULL if the layouts are incompatible, but
now that I think about it, maybe it would be better to allow any
combination, but just skip the reordering and/or downmixing if it
doesn't know how to properly mix the 2 layouts.

> 
> 
>>av_channel_mix does the channel reordering (and will do downmixing once
>>it is implemented).
>>
>>av_channel_mix_close frees the AVChannelMixContext.
>>
>>The attached patch is not fully-functional, but gives an idea of how it
>>might work.  The example used is encoding PCM wav to raw AC3.  The
>>user-level parts (ffmpeg.c and ffplay.c) are not implemented yet, so the
>>patch doesn't work at this point.
>>
>>Also, I don't really know what appropriate channel positions should be.
> 
> 
> just guess and add a note that they are just guessed ...

I'll give it a shot.  As far as format, azimuth needs +/- 180 deg,
elevation needs +/- 90 deg.  So 16-bit signed int + 16-bit fixed-point
should be way more than adequate.


> [...]
> 
>>Index: libavformat/avformat.h
>>===================================================================
>>--- libavformat/avformat.h	(revision 8673)
>>+++ libavformat/avformat.h	(working copy)
>>@@ -193,6 +193,12 @@
>> 
>>     /* private fields */
>>     struct AVOutputFormat *next;
>>+
>>+    /**
>>+     * array of supported channel layouts, or NULL if any
>>+     * array is terminanted by NULL
>>+     */
>>+    const AVChannelLayout **channel_layouts[];
>> } AVOutputFormat;
> 
> 
> you can place this before *next i think as *next is private

Good...I was wondering about that.

> and ia fine with the general design i think ...

ok.  Then I'll go ahead and fill in the details to make it work
properly.  Should I go ahead and implement this in the formats and
codecs I know about or just do the framework first then take it one
format/codec at a time?  If the AVCodecContext->channel_layout is left
NULL it will keep the current behavior.

-Justin




More information about the ffmpeg-devel mailing list