[FFmpeg-devel] [PATCH 9/9] ffplay: add -af option
Marton Balint
cus at passwd.hu
Tue Feb 5 00:35:33 CET 2013
On Sun, 3 Feb 2013, Stefano Sabatini wrote:
> On date Sunday 2013-02-03 16:30:11 +0100, Stefano Sabatini encoded:
> [...]
>> Updated work in progress.
>>
>> I'm not sure about how to handle configuration in open_audio(). Right
>> now I'm configuring the filtergraph, *then* opening the audio device.
>>
>> In order to minimize changes, in audio_decode_frame() (which should be
>> renamed something like audio_process_frame) I'm passing is->frame to
>> the filtergraph, and I convert it back to is->frame if I manage to get
>> a frame. Then the data is handled the usual way.
>>
>> In case the device target configuration differs from the output
>> filter, the normalization code in audio_decode_frame() should convert
>> it to the target configurations (possibly not very efficient).
>> Alternatively: we configure the filters, open the device, then
>> *reconfigure* the filters to take into account of the device
>> configuration (and virtually avoids the need for the output
>> normalization).
>>
>> The current design doesn't address the case where the input changes,
>> which should be addressed by either reconfiguring the filtergraph or
>> adding another normalization layer between the decoder and the
>> filtergraph.
>
> Updated.
>
> This one solves all the above mentioned problems, but it is currently
> skipping sample compensation. The only approach I can see is to use a
> dedicated swr context for that.
>
> Not yet ready for commit, but feel free to comment.
> --
> FFmpeg = Free and Fundamentalist Multimedia Portable Enhancing Gem
>
>From b03a8a5c31dbf600bb55d8381bb5eeac0c0e91e1 Mon Sep 17 00:00:00 2001
>From: Stefano Sabatini <stefasab at gmail.com>
>Date: Sun, 3 Feb 2013 21:29:01 +0100
>Subject: [PATCH] ffplay: add -af option
>
>---
> doc/ffplay.texi | 6 ++
> ffplay.c | 194 +++++++++++++++++++++++++++++++++++++++++++++++++++----
> 2 files changed, 188 insertions(+), 12 deletions(-)
>
>diff --git a/doc/ffplay.texi b/doc/ffplay.texi
>index 21b4683..566bc19 100644
>--- a/doc/ffplay.texi
>+++ b/doc/ffplay.texi
>@@ -84,6 +84,12 @@ output. In the filter graph, the input is associated to the label
> ffmpeg-filters manual for more information about the filtergraph
> syntax.
>
>+ at item -af @var{filter_graph}
>+ at var{filter_graph} is a description of the filter graph to apply to
>+the input audio.
>+Use the option "-filters" to show all the available filters (including
>+sources and sinks).
>+
> @item -i @var{input_file}
> Read @var{input_file}.
> @end table
>diff --git a/ffplay.c b/ffplay.c
>index 199b0f6..2ba37d2 100644
>--- a/ffplay.c
>+++ b/ffplay.c
>@@ -125,6 +125,7 @@ typedef struct VideoPicture {
>
> #if CONFIG_AVFILTER
> AVFilterBufferRef *picref;
>+ AVFilterBufferRef *samplesref;
> #endif
> } VideoPicture;
>
>@@ -254,6 +255,10 @@ typedef struct VideoState {
> AVFilterContext *out_video_filter; // the last filter in the video chain
> int use_dr1;
> FrameBuffer *buffer_pool;
>+
>+ AVFilterContext *in_audio_filter; ///<the first filter in the audio chain
>+ AVFilterContext *out_audio_filter; ///<the last filter in the audio chain
>+ AVFilterGraph *agraph; ///<audio filter graph
> #endif
>
> int refresh;
>@@ -310,6 +315,7 @@ static int64_t cursor_last_shown;
> static int cursor_hidden = 0;
> #if CONFIG_AVFILTER
> static char *vfilters = NULL;
>+static char *afilters = NULL;
> #endif
>
> /* current context */
>@@ -324,6 +330,16 @@ static AVPacket flush_pkt;
>
> static SDL_Surface *screen;
>
>+static inline
>+int cmp_audio_fmts(enum AVSampleFormat fmt1, int64_t channel_count1,
>+ enum AVSampleFormat fmt2, int64_t channel_count2)
>+{
>+ int ret =
>+ channel_count1 == channel_count2 == 1 ? av_get_packed_sample_fmt(fmt1) != av_get_packed_sample_fmt(fmt2) :
>+ fmt1 != fmt2;
Please make this more readable even if it works.
>+ return ret;
>+}
>+
> static int packet_queue_put(PacketQueue *q, AVPacket *pkt);
>
> static int packet_queue_put_private(PacketQueue *q, AVPacket *pkt)
>@@ -1807,6 +1823,72 @@ fail:
> return ret;
> }
>
>+static int configure_audio_filters(VideoState *is, const char *afilters)
>+{
>+ static const enum AVSampleFormat sample_fmts[] = { AV_SAMPLE_FMT_S16, PIX_FMT_NONE };
>+ int sample_rates[2] = { 0, -1 };
>+ int64_t channel_layouts[2] = { 0, -1 };
>+ int channels[2] = { 0, -1 };
>+ AVFilterContext *filt_asrc = NULL, *filt_asink = NULL;
>+ char abuffer_args[256];
>+ AVABufferSinkParams *abuffersink_params = NULL;
>+ int ret;
>+
>+ avfilter_graph_free(&is->agraph);
>+ if (!(is->agraph = avfilter_graph_alloc()))
>+ return AVERROR(ENOMEM);
>+
>+ snprintf(abuffer_args, sizeof(abuffer_args),
>+ "sample_rate=%d:sample_fmt=%s:channel_layout=0x%"PRIx64":channels=%"PRId64,
>+ is->audio_src.freq, av_get_sample_fmt_name(is->audio_src.fmt),
>+ is->audio_src.channel_layout, is->audio_src.channels);
>+ ret = avfilter_graph_create_filter(&filt_asrc,
>+ avfilter_get_by_name("abuffer"), "ffplay_abuffer",
>+ abuffer_args, NULL, is->agraph);
>+ if (ret < 0)
>+ goto fail;
>+
>+ if (!(abuffersink_params = av_abuffersink_params_alloc())) {
>+ ret = AVERROR(ENOMEM);
>+ goto fail;
>+ }
>+ abuffersink_params->sample_fmts = sample_fmts;
>+
>+ abuffersink_params->all_channel_counts = 1;
>+ if (is->audio_tgt.channel_layout != 0) {
>+ channel_layouts[0] = is->audio_tgt.channel_layout;
>+ abuffersink_params->channel_layouts = channel_layouts;
>+ abuffersink_params->all_channel_counts = 0;
>+ }
>+ if (is->audio_tgt.channels != 0) {
>+ channels[0] = is->audio_tgt.channels;
>+ abuffersink_params->channel_counts = channels;
>+ abuffersink_params->all_channel_counts = 0;
>+ }
>+
>+ if (is->audio_tgt.freq) {
>+ sample_rates[0] = is->audio_tgt.freq;
>+ abuffersink_params->sample_rates = sample_rates;
>+ }
I guess you are checking the parameters of audio_tgt against zero for the
default output type detection before audio_open. I think it is more
cleaner if you just add an extra paramter to configure_video_filters to do
that, audio_tgt may have already been set from a previous audio stream...
>+
>+ ret = avfilter_graph_create_filter(&filt_asink,
>+ avfilter_get_by_name("abuffersink"), "ffplay_abuffersink",
>+ NULL, abuffersink_params, is->agraph);
>+ if (ret < 0)
>+ goto fail;
>+
>+ if ((ret = configure_filtergraph(is->agraph, afilters, filt_asrc, filt_asink)) < 0)
>+ goto fail;
>+
>+ is->in_audio_filter = filt_asrc;
>+ is->out_audio_filter = filt_asink;
>+ return 0;
>+
>+fail:
>+ avfilter_graph_free(&is->agraph);
>+ return ret;
>+}
>+
> #endif /* CONFIG_AVFILTER */
>
> static int video_thread(void *arg)
>@@ -2094,13 +2176,11 @@ static int audio_decode_frame(VideoState *is)
> AVPacket *pkt_temp = &is->audio_pkt_temp;
> AVPacket *pkt = &is->audio_pkt;
> AVCodecContext *dec = is->audio_st->codec;
>- int len1, len2, data_size, resampled_data_size;
>- int64_t dec_channel_layout;
>+ int len1, data_size, resampled_data_size;
> int got_frame;
> av_unused double audio_clock0;
> int new_packet = 0;
> int flush_complete = 0;
>- int wanted_nb_samples;
>
> for (;;) {
> /* NOTE: the audio packet can contain several frames */
>@@ -2133,9 +2213,70 @@ static int audio_decode_frame(VideoState *is)
> flush_complete = 1;
> continue;
> }
>+
> data_size = av_samples_get_buffer_size(NULL, is->frame->channels,
> is->frame->nb_samples,
> is->frame->format, 1);
>+#if CONFIG_AVFILTER
>+ {
>+ AVFilterBufferRef *samplesref;
>+ static int first = 1;
>+ int ret;
>+ int reconfigure;
>+
>+ reconfigure =
>+ cmp_audio_fmts(is->audio_src.fmt, is->audio_src.channels,
>+ is->frame->format, is->frame->channels) ||
>+ is->audio_src.channel_layout != is->frame->channel_layout ||
>+ is->audio_src.freq != is->frame->sample_rate;
>+ if (first) {
>+ AVFilterLink *link = is->out_audio_filter->inputs[0];
>+
>+ reconfigure |=
>+ cmp_audio_fmts(is->audio_tgt.fmt, is->audio_tgt.channels,
>+ link->format, link->channels) ||
>+ is->audio_tgt.channel_layout != link->channel_layout ||
>+ is->audio_tgt.freq != link->sample_rate;
>+ }
Instead of using first and checking conditions I think it is much simpler
to do a reconfiguration after audio_open in the CONFIG_AVFILTER
enabled case.
>+
>+ if (reconfigure) {
>+ char buf1[1024], buf2[1024];
>+ av_get_channel_layout_string(buf1, sizeof(buf1), -1, is->audio_src.channel_layout);
>+ av_get_channel_layout_string(buf2, sizeof(buf2), -1, is->frame->channel_layout);
>+ av_log(NULL, AV_LOG_INFO,
>+ "Audio frame changed from rate:%d ch:%"PRId64" fmt:%s layout:%s to rate:%d ch:%"PRId64" fmt:%s layout:%s\n",
>+ is->audio_src.freq, is->audio_src.channels, av_get_sample_fmt_name(is->audio_src.fmt), buf1,
>+ is->frame->sample_rate, is->frame->channels, av_get_sample_fmt_name(is->frame->format), buf2);
>+
>+ is->audio_src.fmt = is->frame->format;
>+ is->audio_src.channels = is->frame->channels;
>+ is->audio_src.channel_layout = is->frame->channel_layout;
>+ is->audio_src.freq = is->frame->sample_rate;
>+
>+ if ((ret = configure_audio_filters(is, afilters)) < 0)
>+ return ret;
>+ }
>+
>+ if (is->frame->pts == AV_NOPTS_VALUE && is->frame->pkt_pts != AV_NOPTS_VALUE)
>+ is->frame->pts = av_rescale_q(is->frame->pkt_pts, is->audio_st->time_base, dec->time_base);
>+ if ((ret = av_buffersrc_add_frame(is->in_audio_filter, is->frame, 0)) < 0)
>+ return ret;
>+ if ((ret = av_buffersink_get_buffer_ref(is->out_audio_filter,
>+ &samplesref, 0)) < 0)
>+ return ret;
>+ if ((ret = avfilter_copy_buf_props(is->frame, samplesref)) < 0)
>+ return ret;
>+ first = 0;
>+ }
>+
>+ resampled_data_size = av_samples_get_buffer_size(NULL, is->frame->channels,
>+ is->frame->nb_samples,
>+ is->frame->format, 1);
>+ is->audio_buf = is->frame->data[0];
>+#else
>+ {
>+ int wanted_nb_samples, len2;
>+ int64_t dec_channel_layout;
>
> dec_channel_layout =
> (is->frame->channel_layout && is->frame->channels == av_get_channel_layout_nb_channels(is->frame->channel_layout)) ?
>@@ -2193,6 +2334,8 @@ static int audio_decode_frame(VideoState *is)
> is->audio_buf = is->frame->data[0];
> resampled_data_size = data_size;
> }
>+ }
>+#endif
>
> audio_clock0 = is->audio_clock;
> is->audio_clock += (double)data_size /
>@@ -2343,6 +2486,9 @@ static int stream_component_open(VideoState *is, int stream_index)
> const char *forced_codec_name = NULL;
> AVDictionary *opts;
> AVDictionaryEntry *t = NULL;
>+ int sample_rate, nb_channels;
>+ int64_t channel_layout;
>+ int ret;
>
> if (stream_index < 0 || stream_index >= ic->nb_streams)
> return -1;
>@@ -2392,15 +2538,6 @@ static int stream_component_open(VideoState *is, int stream_index)
> return AVERROR_OPTION_NOT_FOUND;
> }
>
>- /* prepare audio output */
>- if (avctx->codec_type == AVMEDIA_TYPE_AUDIO) {
>- int audio_hw_buf_size = audio_open(is, avctx->channel_layout, avctx->channels, avctx->sample_rate, &is->audio_src);
>- if (audio_hw_buf_size < 0)
>- return -1;
>- is->audio_hw_buf_size = audio_hw_buf_size;
>- is->audio_tgt = is->audio_src;
>- }
>-
> ic->streams[stream_index]->discard = AVDISCARD_DEFAULT;
> switch (avctx->codec_type) {
> case AVMEDIA_TYPE_AUDIO:
>@@ -2418,6 +2555,35 @@ static int stream_component_open(VideoState *is, int stream_index)
>
> memset(&is->audio_pkt, 0, sizeof(is->audio_pkt));
> memset(&is->audio_pkt_temp, 0, sizeof(is->audio_pkt_temp));
>+
>+#if CONFIG_AVFILTER
>+ {
>+ AVFilterLink *link;
>+
>+ is->audio_src.freq = avctx->sample_rate;
>+ is->audio_src.channels = avctx->channels;
>+ is->audio_src.channel_layout = avctx->channel_layout;
>+ is->audio_src.fmt = avctx->sample_fmt;
>+ if ((ret = configure_audio_filters(is, afilters)) < 0)
>+ return ret;
>+ link = is->out_audio_filter->inputs[0];
>+ sample_rate = link->sample_rate;
>+ nb_channels = link->channels;
>+ channel_layout = link->channel_layout;
>+ }
>+#else
>+ sample_rate = avctx->sample_rate;
>+ nb_channels = avctx->channels;
>+ channel_layout = avctx->channel_layout;
>+#endif
>+
>+ /* prepare audio output */
>+ if ((ret = audio_open(is, channel_layout, nb_channels, sample_rate, &is->audio_tgt)) < 0)
>+ return ret;
>+ is->audio_hw_buf_size = ret;
>+ if (!CONFIG_AVFILTER)
>+ is->audio_src = is->audio_tgt;
>+
You moved some code with this modification, but above this block there is
code which is using the result of this block (audio_diff_threshold
calculation). So move more code, or do not move at all :)
> packet_queue_start(&is->audioq);
> SDL_PauseAudio(0);
> break;
>@@ -2470,6 +2636,9 @@ static void stream_component_close(VideoState *is, int stream_index)
> is->rdft = NULL;
> is->rdft_bits = 0;
> }
>+#if CONFIG_AVFILTER
>+ avfilter_graph_free(&is->agraph);
>+#endif
> break;
> case AVMEDIA_TYPE_VIDEO:
> packet_queue_abort(&is->videoq);
>@@ -3250,6 +3419,7 @@ static const OptionDef options[] = {
> { "window_title", OPT_STRING | HAS_ARG, { &window_title }, "set window title", "window title" },
> #if CONFIG_AVFILTER
> { "vf", OPT_STRING | HAS_ARG, { &vfilters }, "set video filters", "filter_graph" },
>+ { "af", OPT_STRING | HAS_ARG, { &afilters }, "set audio filters", "filter_graph" },
> #endif
> { "rdftspeed", OPT_INT | HAS_ARG| OPT_AUDIO | OPT_EXPERT, { &rdftspeed }, "rdft speed", "msecs" },
> { "showmode", HAS_ARG, { .func_arg = opt_show_mode}, "select show mode (0 = video, 1 = waves, 2 = RDFT)", "mode" },
>--
>1.7.9.5
>
I haven't looked thoroughly, but I think it should not be too hard to
handle the compensation with the current swresampler code which was there
for the resampling. (With your patch, it is only enabled now in the
AVFILTER disabled case). So I guess you need to enable that code path for
all cases, and for keeping track of audio input parameters for filtering,
you should use a different variable (e.g.: VideoState->audio_filt_src), so
VideoState->audio_src can be used the way it was used before.
Regards,
Marton
More information about the ffmpeg-devel
mailing list