<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Title" content="">
<meta name="Keywords" content="">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:Calibri;
mso-fareast-language:EN-US;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:#0563C1;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
span.EmailStyle17
{mso-style-type:personal;
font-family:Calibri;
color:windowtext;}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:595.0pt 842.0pt;
margin:70.85pt 3.0cm 70.85pt 3.0cm;}
div.WordSection1
{page:WordSection1;}
--></style>
</head>
<body bgcolor="white" lang="EN-GB" link="#0563C1" vlink="#954F72">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt">Hi everyone!<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt">The error was in another part of the code. This audio filter just works fine.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt"><o:p> </o:p></span></p>
<div>
<p class="MsoNormal"><b><span lang="ES" style="mso-fareast-language:EN-GB">Rubén Sánchez Castellano<o:p></o:p></span></b></p>
<p class="MsoNormal"><i><span lang="ES" style="mso-fareast-language:EN-GB">Senior Programmer<o:p></o:p></span></i></p>
<p class="MsoNormal"><span lang="ES" style="mso-fareast-language:EN-GB">Telescope Inc Barcelona<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><span lang="ES" style="font-size:11.0pt"><o:p> </o:p></span></p>
<p class="MsoNormal"><span lang="ES" style="font-size:11.0pt"><o:p> </o:p></span></p>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b><span style="color:black">From: </span></b><span style="color:black">Libav-user <libav-user-bounces@ffmpeg.org> on behalf of Rubén Sánchez <ruben.sanchez@telescope.tv><br>
<b>Reply-To: </b>"This list is about using libavcodec, libavformat, libavutil, libavdevice and libavfilter." <libav-user@ffmpeg.org><br>
<b>Date: </b>Friday, 11 August 2017 at 14:11<br>
<b>To: </b>"libav-user@ffmpeg.org" <libav-user@ffmpeg.org><br>
<b>Subject: </b>[Libav-user] Audio sync issue when resampling audio with libavfilter</span><span style="color:black;mso-fareast-language:EN-GB"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Times New Roman""><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt">Hi!</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">I'm developing an app that among other features, must resample 48000KHz audio input into 44100 audio output and encode it using AAC encoder. I'm able to resample the audio with the code attached but after
30 minutes the audio gets out of sync and the audio goes after the video. This particular case has the following context:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Input: Stereo, FLTP, 48KHz</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Output: Stereo, FLTP, 44.1KHz</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">FFmpeg version 3.3.3</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">This is the method I use to initialize the filter:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">================= CODE ========================</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">AVFilterGraph *filter_graph = NULL;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">AVFilterContext *buffersrc_ctx = NULL;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">AVFilterContext *buffersink_ctx = NULL;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">int initialize_audio_filter(AVStream *inputStream) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> char args[512];</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> int ret;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AVFilter *buffersrc = avfilter_get_by_name("abuffer");</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AVFilter *buffersink = avfilter_get_by_name("abuffersink");</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AVFilterInOut *outputs = avfilter_inout_alloc();</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AVFilterInOut *inputs = avfilter_inout_alloc();</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> filter_graph = avfilter_graph_alloc();</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> const enum AVSampleFormat out_sample_fmts[] = {AV_SAMPLE_FMT_FLTP, AV_SAMPLE_FMT_NONE};</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> const int64_t out_channel_layouts[] = {AV_CH_LAYOUT_STEREO, -1};</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> const int out_sample_rates[] = {44100, -1};</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> snprintf(args, sizeof(args), "time_base=%d/%d:sample_rate=%d:sample_fmt=%s:channel_layout=0x%" PRIx64,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> inputStream->codec->time_base.num, inputStream->codec->time_base.den,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> inputStream->codec->sample_rate,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> av_get_sample_fmt_name(inputStream->codec->sample_fmt),</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> inputStream->codec->channel_layout);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ret = avfilter_graph_create_filter(&buffersrc_ctx, buffersrc, "in", args, NULL, filter_graph);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if (ret < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsCritical("", QString("Could not create filter graph, error: %1").arg(svsAvErrorToFormattedString(ret)))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return -1;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ret = avfilter_graph_create_filter(&buffersink_ctx, buffersink, "out", NULL, NULL, filter_graph);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if (ret < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsCritical("", QString("Cannot create buffer sink, error: %1").arg(svsAvErrorToFormattedString(ret)))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return ret;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ret = av_opt_set_int_list(buffersink_ctx, "sample_fmts", out_sample_fmts, -1,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AV_OPT_SEARCH_CHILDREN);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if (ret < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsCritical("", QString("Cannot set output sample format, error: %1").arg(svsAvErrorToFormattedString(ret)))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return ret;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ret = av_opt_set_int_list(buffersink_ctx, "channel_layouts", out_channel_layouts, -1,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AV_OPT_SEARCH_CHILDREN);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if (ret < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsCritical("", QString("Cannot set output channel layout, error: %1").arg(svsAvErrorToFormattedString(ret)))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return ret;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ret = av_opt_set_int_list(buffersink_ctx, "sample_rates", out_sample_rates, -1,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AV_OPT_SEARCH_CHILDREN);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if (ret < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsCritical("", QString("Cannot set output sample rate, error: %1").arg(svsAvErrorToFormattedString(ret)))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return ret;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> /* Endpoints for the filter graph. */</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> outputs -> name = av_strdup("in");</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> outputs -> filter_ctx = buffersrc_ctx;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> outputs -> pad_idx = 0;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> outputs -> next = NULL;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> /* Endpoints for the filter graph. */</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> inputs -> name = av_strdup("out");</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> inputs -> filter_ctx = buffersink_ctx;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> inputs -> pad_idx = 0;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> inputs -> next = NULL;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> QString filter_description = "aresample=44100:async=100,asetnsamples=n=1024:p=0,aformat=sample_fmts=fltp:channel_layouts=stereo";</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if ((ret = avfilter_graph_parse_ptr(filter_graph, filter_description.toStdString().c_str(), &inputs, &outputs, NULL)) < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsCritical("", QString("Could not add the filter to graph, error: %1").arg(svsAvErrorToFormattedString(ret)))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if ((ret = avfilter_graph_config(filter_graph, NULL)) < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsCritical("", QString("Could not configure the graph, error: %1").arg(svsAvErrorToFormattedString(ret)))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> /* Print summary of the sink buffer</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> * Note: args buffer is reused to store channel layout string */</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AVFilterLink *outlink = buffersink_ctx->inputs[0];</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> av_get_channel_layout_string(args, sizeof(args), -1, outlink->channel_layout);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsInfo("", QString::asprintf("Output: srate:%dHz fmt:%s chlayout:%s\n",</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> (int) outlink->sample_rate,</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> (char *) av_x_if_null(av_get_sample_fmt_name((AVSampleFormat) outlink->format), "?"),</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> args))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return 0;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">================= END CODE ========================</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Notice I configure the audio filter to generate the output with 1024 samples since this is the amount of samples AAC needs to encode an audio frame.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">And this is how I use it:</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">================= CODE ========================</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">AVFrame* StreamFeedController::resampleAudio(const QString& key, AVFrame *frame) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> /* Push the decoded frame into the filtergraph */</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> qint32 ret;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ret = av_buffersrc_add_frame_flags(buffersrc_ctx, frame, AV_BUFFERSRC_FLAG_KEEP_REF);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if(ret < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> svsWarning(key, QString("Error adding frame to buffer: %1").arg(svsAvErrorToFormattedString(ret)))</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> // Delete input frame and return null</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> av_frame_unref(frame);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return nullptr;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> AVFrame *resampled_frame = av_frame_alloc();</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> /* Pull filtered frames from the filtergraph */</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> ret = av_buffersink_get_frame(buffersink_ctx, resampled_frame);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> /* Set the timestamp on the resampled frame */</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">// resampled_frame->pts = frame->pts;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> resampled_frame->best_effort_timestamp = resampled_frame->pts;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> if(ret < 0) {</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> // This is very common. For 48KHz -> 44.1KHz for some input frames the</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> // filter has not data enough to generate another output.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> av_frame_unref(frame);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> av_frame_unref(resampled_frame);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return nullptr;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> }</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> av_frame_unref(frame);</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> return resampled_frame;</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">}</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">================= END CODE ========================</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt">Any idea why the filter stops working after 30 minutes? I could reproduce this always.</span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><span style="font-size:11.0pt"> </span><o:p></o:p></p>
<p class="MsoNormal"><b><span lang="ES" style="mso-fareast-language:EN-GB">Rubén Sánchez Castellano</span></b><o:p></o:p></p>
<p class="MsoNormal"><i><span lang="ES" style="mso-fareast-language:EN-GB">Senior Programmer</span></i><o:p></o:p></p>
<p class="MsoNormal"><span lang="ES" style="mso-fareast-language:EN-GB">Telescope Inc Barcelona</span><o:p></o:p></p>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</body>
</html>