[FFmpeg-devel] [PATCH 2/4] pan: add channel mapping capability.

Nicolas George nicolas.george at normalesup.org
Mon Jan 23 15:05:53 CET 2012


Le quartidi 4 pluviôse, an CCXX, Clément Bœsch a écrit :
> I meant "remap == gains are pure"; remap being opposed to panning by its
> purity.

But gains are pure... what? Pure water? Pure integers?

>	  This debate is slightly moving to a philosophical bikeshed

Of course.

> libswresample just keeps a pointer to the channel mapping array internally, so
> at the moment it is required. This could be changed eventually.

I stand corrected.

> Mmh, I can't check it here, so I added a check in config_props() callback.  But
> I'm not sure the checks are really necessary actually, since AFAIK no existing
> channel layouts have more channels than SWR_CH_MAX.

Well, I guess we could state that SWR_CH_MAX is an absolute maximum for the
number of channels lavfi can accept, and raise it if it becomes necessary.

> Well, it looks like a standard function call to me:

Exactly: with a branch instead of a function pointer, it could be a simple
jump, saving all the registers saving/restoring and arguments passing.

>  - bumped lavfi micro

Beware, git rebase will silently assume the hunk has been cherry-picked
instead of showing a conflict.

> From 95f57535fc5f418173502a52bdbdeb0def4fbc20 Mon Sep 17 00:00:00 2001
> From: =?UTF-8?q?Cl=C3=A9ment=20B=C5=93sch?= <clement.boesch at smartjog.com>
> Date: Wed, 18 Jan 2012 12:00:16 +0100
> Subject: [PATCH] pan: add channel mapping capability.
> 
> ---
>  configure             |    1 +
>  doc/filters.texi      |   45 +++++++++++++++++
>  libavfilter/af_pan.c  |  133 ++++++++++++++++++++++++++++++++++++++++++++++--
>  libavfilter/version.h |    2 +-
>  4 files changed, 174 insertions(+), 7 deletions(-)
> 
> diff --git a/configure b/configure
> index 2a3eaa1..a06e63c 100755
> --- a/configure
> +++ b/configure
> @@ -1659,6 +1659,7 @@ mp_filter_deps="gpl avcodec"
>  mptestsrc_filter_deps="gpl"
>  negate_filter_deps="lut_filter"
>  ocv_filter_deps="libopencv"
> +pan_filter_deps="swresample"
>  scale_filter_deps="swscale"
>  tinterlace_filter_deps="gpl"
>  yadif_filter_deps="gpl"
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 3c9f554..7d008bc 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -315,6 +315,9 @@ Ported from SoX.
>  Mix channels with specific gain levels. The filter accepts the output
>  channel layout followed by a set of channels definitions.
>  
> +This filter is also designed to remap efficiently the channels of an audio
> +stream.
> +
>  The filter accepts parameters of the form:
>  "@var{l}:@var{outdef}:@var{outdef}:..."
>  
> @@ -342,6 +345,8 @@ If the `=' in a channel specification is replaced by `<', then the gains for
>  that specification will be renormalized so that the total is 1, thus
>  avoiding clipping noise.
>  
> + at subsection Mixing examples
> +
>  For example, if you want to down-mix from stereo to mono, but with a bigger
>  factor for the left channel:
>  @example
> @@ -358,6 +363,46 @@ Note that @command{ffmpeg} integrates a default down-mix (and up-mix) system
>  that should be preferred (see "-ac" option) unless you have very specific
>  needs.
>  
> + at subsection Remapping examples
> +
> +The channel remapping will be effective if, and only if:
> +
> + at itemize
> + at item gain coefficients are zeroes or ones,
> + at item only one input per channel output,
> + at item the number of output channels is supported by libswresample (16 at the
> +      moment)
> + at c if SWR_CH_MAX changes, fix the line above.
> + at end itemize
> +
> +If all these conditions are satisfied, the filter will notify the user ("Pure
> +channel mapping detected"), and use an optimized and lossless method to do the
> +remapping.
> +
> +For example, if you have a 5.1 source and want a stereo audio stream by
> +dropping the extra channels:
> + at example
> +pan="stereo: c0=FL : c1=FR"
> + at end example
> +
> +Given the same source, you can also switch front left and front right channels
> +and keep the input channel layout:
> + at example
> +pan="5.1: c0=c1 : c1=c0 : c2=c2 : c3=c3 : c4=c4 : c5=c5"
> + at end example
> +
> +If the input is a stereo audio stream, you can mute the front left channel (and
> +still keep the stereo channel layout) with:
> + at example
> +pan="stereo:c1=c1"
> + at end example
> +
> +Still with a stereo audio stream input, you can copy the right channel in both
> +front left and right:
> + at example
> +pan="stereo: c0=FR : c1=FR"
> + at end example
> +
>  @section silencedetect
>  
>  Detect silence in an audio stream.
> diff --git a/libavfilter/af_pan.c b/libavfilter/af_pan.c
> index a62dea1..d6bb0dc 100644
> --- a/libavfilter/af_pan.c
> +++ b/libavfilter/af_pan.c
> @@ -30,12 +30,14 @@
>  #include <stdio.h>
>  #include "libavutil/audioconvert.h"
>  #include "libavutil/avstring.h"
> +#include "libavutil/opt.h"
> +#include "libswresample/swresample.h"
>  #include "avfilter.h"
>  #include "internal.h"
>  
>  #define MAX_CHANNELS 63
>  
> -typedef struct {
> +typedef struct PanContext {
>      int64_t out_channel_layout;
>      union {
>          double d[MAX_CHANNELS][MAX_CHANNELS];
> @@ -46,6 +48,16 @@ typedef struct {
>      int need_renumber;
>      int nb_input_channels;
>      int nb_output_channels;
> +
> +    int pure_gains;
> +    void (*filter_samples)(struct PanContext*,
> +                           AVFilterBufferRef*,
> +                           AVFilterBufferRef*,
> +                           int);
> +
> +    /* channel mapping specific */
> +    int channel_map[SWR_CH_MAX];
> +    struct SwrContext *swr;
>  } PanContext;
>  
>  static int parse_channel_name(char **arg, int *rchannel, int *rnamed)
> @@ -179,6 +191,31 @@ static av_cold int init(AVFilterContext *ctx, const char *args0, void *opaque)
>      return 0;
>  }
>  
> +static void filter_samples_channel_mapping(PanContext *pan, AVFilterBufferRef *outsamples, AVFilterBufferRef *insamples, int n);
> +static void filter_samples_panning        (PanContext *pan, AVFilterBufferRef *outsamples, AVFilterBufferRef *insamples, int n);
> +
> +static int are_gains_pure(const PanContext *pan)
> +{
> +    int i, j;
> +
> +    for (i = 0; i < MAX_CHANNELS; i++) {
> +        int nb_gain = 0;
> +
> +        for (j = 0; j < MAX_CHANNELS; j++) {
> +            double gain = pan->gain.d[i][j];
> +
> +            /* channel mapping is effective only if 0% or 100% of a channel is
> +             * selected... */
> +            if (gain != 0. && gain != 1.)
> +                return 0;
> +            /* ...and if the output channel is only composed of one input */
> +            if (gain && nb_gain++)
> +                return 0;
> +        }
> +    }
> +    return 1;
> +}
> +
>  static int query_formats(AVFilterContext *ctx)
>  {
>      PanContext *pan = ctx->priv;
> @@ -186,11 +223,21 @@ static int query_formats(AVFilterContext *ctx)
>      AVFilterLink *outlink = ctx->outputs[0];
>      AVFilterFormats *formats;
>  
> +    if (pan->nb_output_channels <= SWR_CH_MAX)
> +        pan->pure_gains = are_gains_pure(pan);
> +    if (pan->pure_gains) {
> +        /* libswr supports any sample and packing formats */
> +        avfilter_set_common_sample_formats(ctx, avfilter_make_all_formats(AVMEDIA_TYPE_AUDIO));
> +        avfilter_set_common_packing_formats(ctx, avfilter_make_all_packing_formats());
> +        pan->filter_samples = filter_samples_channel_mapping;
> +    } else {
>      const enum AVSampleFormat sample_fmts[] = {AV_SAMPLE_FMT_S16, -1};
>      const int                packing_fmts[] = {AVFILTER_PACKED,   -1};
>  
>      avfilter_set_common_sample_formats (ctx, avfilter_make_format_list(sample_fmts));
>      avfilter_set_common_packing_formats(ctx, avfilter_make_format_list(packing_fmts));
> +    pan->filter_samples = filter_samples_panning;
> +    }
>  
>      // inlink supports any channel layout
>      formats = avfilter_make_all_channel_layouts();
> @@ -222,6 +269,44 @@ static int config_props(AVFilterLink *link)
>              }
>          }
>      }
> +    // gains are pure, init the channel mapping
> +    if (pan->pure_gains) {
> +
> +        // sanity check; can't be done in query_formats since the inlink
> +        // channel layout is unknown at that time
> +        if (pan->nb_input_channels >= SWR_CH_MAX) {

I believe nb_input_channels == SWR_CH_MAX should be accepted.

> +            av_log(ctx, AV_LOG_ERROR,
> +                   "libswresample support a maximum of %d channels. "
> +                   "Feel free to ask for a higher limit.\n", SWR_CH_MAX);
> +            return AVERROR_PATCHWELCOME;
> +        }
> +
> +        // get channel map from the pure gains
> +        for (i = 0; i < pan->nb_output_channels; i++) {
> +            int ch_id = -1;
> +            for (j = 0; j < pan->nb_input_channels; j++) {
> +                if (pan->gain.d[i][j]) {
> +                    ch_id = j;
> +                    break;
> +                }
> +            }
> +            pan->channel_map[i] = ch_id;
> +        }
> +
> +        // init libswresample context
> +        pan->swr = swr_alloc_set_opts(pan->swr,
> +                                      pan->out_channel_layout, link->format, link->sample_rate,
> +                                      link->channel_layout,    link->format, link->sample_rate,
> +                                      0, ctx);
> +        if (!pan->swr)
> +            return AVERROR(ENOMEM);
> +        av_opt_set_int(pan->swr, "icl", pan->out_channel_layout, 0);
> +        av_opt_set_int(pan->swr, "uch", pan->nb_output_channels, 0);
> +        swr_set_channel_mapping(pan->swr, pan->channel_map);
> +        r = swr_init(pan->swr);
> +        if (r < 0)
> +            return r;
> +    } else {
>      // renormalize
>      for (i = 0; i < pan->nb_output_channels; i++) {
>          if (!((pan->need_renorm >> i) & 1))
> @@ -239,6 +324,7 @@ static int config_props(AVFilterLink *link)
>          for (j = 0; j < pan->nb_input_channels; j++)
>              pan->gain.d[i][j] /= t;
>      }
> +    }
>      // summary
>      for (i = 0; i < pan->nb_output_channels; i++) {
>          cur = buf;
> @@ -249,6 +335,17 @@ static int config_props(AVFilterLink *link)
>          }
>          av_log(ctx, AV_LOG_INFO, "o%d = %s\n", i, buf);
>      }
> +    // add channel mapping summary if possible
> +    if (pan->pure_gains) {
> +        av_log(ctx, AV_LOG_INFO, "Pure channel mapping detected:");
> +        for (i = 0; i < pan->nb_output_channels; i++)
> +            if (pan->channel_map[i] < 0)
> +                av_log(ctx, AV_LOG_INFO, " M");
> +            else
> +                av_log(ctx, AV_LOG_INFO, " %d", pan->channel_map[i]);
> +        av_log(ctx, AV_LOG_INFO, "\n");
> +        return 0;
> +    }
>      // convert to integer
>      for (i = 0; i < pan->nb_output_channels; i++) {
>          for (j = 0; j < pan->nb_input_channels; j++) {
> @@ -261,19 +358,26 @@ static int config_props(AVFilterLink *link)
>      return 0;
>  }
>  
> +static void filter_samples_channel_mapping(struct PanContext *pan,


struct is not necessary (but not evil either).

> +                                           AVFilterBufferRef *outsamples,
> +                                           AVFilterBufferRef *insamples,
> +                                           int n)
> +{
> +    swr_convert(pan->swr, outsamples->data, n, (void *)insamples->data, n);
> +}
>  
> -static void filter_samples(AVFilterLink *inlink, AVFilterBufferRef *insamples)
> +static void filter_samples_panning(struct PanContext *pan,

Idem.

> +                                   AVFilterBufferRef *outsamples,
> +                                   AVFilterBufferRef *insamples,
> +                                   int n)
>  {
> -    PanContext *const pan = inlink->dst->priv;
> -    int i, o, n = insamples->audio->nb_samples;
> +    int i, o;
>  
>      /* input */
>      const int16_t *in     = (int16_t *)insamples->data[0];
>      const int16_t *in_end = in + n * pan->nb_input_channels;
>  
>      /* output */
> -    AVFilterLink *const outlink = inlink->dst->outputs[0];
> -    AVFilterBufferRef *outsamples = avfilter_get_audio_buffer(outlink, AV_PERM_WRITE, n);
>      int16_t *out = (int16_t *)outsamples->data[0];
>  
>      for (; in < in_end; in += pan->nb_input_channels) {
> @@ -284,16 +388,33 @@ static void filter_samples(AVFilterLink *inlink, AVFilterBufferRef *insamples)
>              *(out++) = v >> 8;
>          }
>      }
> +}
> +
> +static void filter_samples(AVFilterLink *inlink, AVFilterBufferRef *insamples)
> +{
> +    int n = insamples->audio->nb_samples;
> +    AVFilterLink *const outlink = inlink->dst->outputs[0];
> +    AVFilterBufferRef *outsamples = avfilter_get_audio_buffer(outlink, AV_PERM_WRITE, n);
> +    PanContext *pan = inlink->dst->priv;
> +
> +    pan->filter_samples(pan, outsamples, insamples, n);
>  
>      avfilter_filter_samples(outlink, outsamples);
>      avfilter_unref_buffer(insamples);
>  }
>  
> +static av_cold void uninit(AVFilterContext *ctx)
> +{
> +    PanContext *pan = ctx->priv;
> +    swr_free(&pan->swr);
> +}
> +
>  AVFilter avfilter_af_pan = {
>      .name          = "pan",
>      .description   = NULL_IF_CONFIG_SMALL("Remix channels with coefficients (panning)."),
>      .priv_size     = sizeof(PanContext),
>      .init          = init,
> +    .uninit        = uninit,
>      .query_formats = query_formats,
>  
>      .inputs    = (const AVFilterPad[]) {
> diff --git a/libavfilter/version.h b/libavfilter/version.h
> index d2af525..6e19dd7 100644
> --- a/libavfilter/version.h
> +++ b/libavfilter/version.h
> @@ -30,7 +30,7 @@
>  
>  #define LIBAVFILTER_VERSION_MAJOR  2
>  #define LIBAVFILTER_VERSION_MINOR 59
> -#define LIBAVFILTER_VERSION_MICRO 101
> +#define LIBAVFILTER_VERSION_MICRO 102
>  
>  #define LIBAVFILTER_VERSION_INT AV_VERSION_INT(LIBAVFILTER_VERSION_MAJOR, \
>                                                 LIBAVFILTER_VERSION_MINOR, \
> -- 
> 1.7.8.3
> 

It looks good. Unless someone else has remarks, I believe it can be applied.
And no need to send another patch just to replace ">=" by ">" or remove
struct.

Thanks for your work.

Regards,

-- 
  Nicolas George


More information about the ffmpeg-devel mailing list