<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Jul 20, 2017 at 1:19 PM, Anton Shekhovtsov <span dir="ltr"><<a href="mailto:shekh.anton@gmail.com" target="_blank">shekh.anton@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><br><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="gmail-h5">2017-07-19 20:54 GMT+03:00 Kerry Loux <span dir="ltr"><<a href="mailto:louxkr@gmail.com" target="_blank">louxkr@gmail.com</a>></span>:<br></div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div><div class="gmail-h5"><div dir="ltr">Hello all,<div><br></div><div>I have an application where I am opening an audio file that was sampled at 44100 Hz, decoding it, resampling to 16000 Hz, encoding it again (AAC) then broadcasting it on an RTSP stream. On the receiving end, I decode the incoming AAC packets and render them.</div><div><br></div><div>The rendered audio is very slow.</div><div><br></div><div>It appears to me that the problem is related to the AVFrame.nb_samples field. When I read a packet from file (using av_read_frame()), the packet size is 1024 samples (at 44100 Hz). After I resample to 16000 Hz, I have ~1/3 the samples that I had in the original frame (as expected). Then, the frame gets encoded, streamed and decoded. After decoding, the AVFrame.nb_samples is 1024 when I expect it to be 372 or so. The AVCodecContext passed to avcodec_receive_frame() has frame_size = 1024, so I assume that the decoder is setting the number of samples of the decoded frame to 1024 regardless of the number of samples actually contained in the input packet? Or maybe it's my job to ensure that the input packets always contain 1024 samples?</div><div><br></div><div>I'm not entirely sure what's going on. My thoughts include:</div><div>- Try buffering 3x number of input frames prior to resampling so the resulting frame will be ~1024 samples</div><div>- Calculate the number of samples manually (how to do this is unclear) and override the number of samples assigned by the decoder (this seems wrong...)</div><div><br></div><div>Any recommendations? Can I just stick multiple frames together in a larger buffer prior to resampling (i.e. calling swr_convert())?</div><div><br></div><div>Thanks,</div><div><br></div><div>Kerry</div></div>
<br></div></div>______________________________<wbr>_________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org" target="_blank">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" rel="noreferrer" target="_blank">http://ffmpeg.org/mailman/list<wbr>info/libav-user</a><br>
<br></blockquote></div><br></div><div class="gmail_extra">Try to study examples (resampling_audio, transcoding_audio, don't remember which is most relevant).</div><div class="gmail_extra">You are not supposed to resample individual frames. You must feed it continuously. AFAIK this is clearly explained in swr docs.</div><div class="gmail_extra">AAC wants packets of fixed size (1024).</div><div class="gmail_extra"><br></div></div>
<br>______________________________<wbr>_________________<br>
Libav-user mailing list<br>
<a href="mailto:Libav-user@ffmpeg.org">Libav-user@ffmpeg.org</a><br>
<a href="http://ffmpeg.org/mailman/listinfo/libav-user" rel="noreferrer" target="_blank">http://ffmpeg.org/mailman/<wbr>listinfo/libav-user</a><br>
<br></blockquote><div> </div></div>Yes, I am feeding it continuously. I am doing this:</div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_extra">AVPacket* ADTSEncoderInterface::EncodeAudio(const AVFrame& inputFrame)</div><div class="gmail_extra">{</div><div class="gmail_extra"><span style="white-space:pre"> </span>if (avcodec_send_frame(encoderContext, &inputFrame) != 0)</div><div class="gmail_extra"><span style="white-space:pre"> </span>return nullptr;</div><div class="gmail_extra"><br></div><div class="gmail_extra"><span style="white-space:pre"> </span>AVPacket* lastOutputPacket, *nextOutputPacket(nullptr);</div><div class="gmail_extra"><span style="white-space:pre"> </span>bool nextPacketIsA(true);</div><div class="gmail_extra"><span style="white-space:pre"> </span>int returnCode;</div><div class="gmail_extra"><span style="white-space:pre"> </span>do</div><div class="gmail_extra"><span style="white-space:pre"> </span>{</div><div class="gmail_extra"><span style="white-space:pre"> </span>lastOutputPacket = nextOutputPacket;</div><div class="gmail_extra"><span style="white-space:pre"> </span>nextPacketIsA = !nextPacketIsA;</div><div class="gmail_extra"><span style="white-space:pre"> </span>if (nextPacketIsA)</div><div class="gmail_extra"><span style="white-space:pre"> </span>nextOutputPacket = &outputPacketA;</div><div class="gmail_extra"><span style="white-space:pre"> </span>else</div><div class="gmail_extra"><span style="white-space:pre"> </span>nextOutputPacket = &outputPacketB;</div><div class="gmail_extra"><br></div><div class="gmail_extra"><span style="white-space:pre"> </span>returnCode = avcodec_receive_packet(encoderContext, nextOutputPacket);</div><div class="gmail_extra"><span style="white-space:pre"> </span>} while (returnCode == 0);</div><div class="gmail_extra"><br></div><div class="gmail_extra"><span style="white-space:pre"> </span>if (returnCode != AVERROR(EAGAIN) || !lastOutputPacket)</div><div class="gmail_extra"><span style="white-space:pre"> </span>return nullptr;</div><div class="gmail_extra"><br></div><div class="gmail_extra"><span style="white-space:pre"> </span>return lastOutputPacket;</div><div class="gmail_extra">}</div><div class="gmail_extra"><br></div><div class="gmail_extra">I assumed (possibly incorrectly) that if AAC requires packets containing 1024 samples, that I would get AVERROR(EAGAIN) returned from avcodec_receive_packet() if there were not enough input samples available. It seems that this is not the case, however, instead I need to do something myself in order to ensure the encoder has at least 1024 samples before I call avcodec_receive_packet().</div><div class="gmail_extra"><br></div><div class="gmail_extra">I haven't found anything in the documentation to suggest that it is the callers responsibility to do this. Maybe this wouldn't be found in FFmpeg docs, but in documentation describing the AAC format? If that were the case, it may have been helpful if the call to avcodec_send_frame() failed with some kind of "wrong number of input samples" error.</div><div class="gmail_extra"><br></div><div class="gmail_extra">I did find a solution, although it seems rather inefficient. I introduced an additional AVFrame object, fullSizeFrame, and prior to calling the encoder (my EncodeAudio method pasted above), I do this:</div><div class="gmail_extra"><br></div><div class="gmail_extra"><div class="gmail_extra"><span style="white-space:pre"> </span>while (fullSizeFrame->nb_samples < packetSampleCount)// packetSampleCount == 1024</div><div class="gmail_extra"><span style="white-space:pre"> </span>{</div><div class="gmail_extra"><span style="white-space:pre"> </span>assert(!dataQueue.empty());</div><div class="gmail_extra"><span style="white-space:pre"> </span>nextFrame = dataQueue.front();</div><div class="gmail_extra"><span style="white-space:pre"> </span>if (!nextFrame)</div><div class="gmail_extra"><span style="white-space:pre"> </span>continue;</div><div class="gmail_extra"><br></div><div class="gmail_extra"><span style="white-space:pre"> </span>const int samplesToCopy(std::min(packetSampleCount - fullSizeFrame->nb_samples, nextFrame->nb_samples));</div><div class="gmail_extra"><span style="white-space:pre"> </span>memcpy(fullSizeFrame->data[0] + fullSizeFrame->nb_samples * sampleSize, nextFrame->data[0], samplesToCopy * sampleSize);</div><div class="gmail_extra"><span style="white-space:pre"> </span>fullSizeFrame->nb_samples += samplesToCopy;</div><div class="gmail_extra"><span style="white-space:pre"> </span>pendingSamples -= samplesToCopy;</div><div class="gmail_extra"><br></div><div class="gmail_extra"><span style="white-space:pre"> </span>if (samplesToCopy == nextFrame->nb_samples)</div><div class="gmail_extra"><span style="white-space:pre"> </span>{</div><div class="gmail_extra"><span style="white-space:pre"> </span>dataQueue.pop();</div><div class="gmail_extra"><span style="white-space:pre"> </span>av_frame_free(&nextFrame);</div><div class="gmail_extra"><span style="white-space:pre"> </span>}</div><div class="gmail_extra"><span style="white-space:pre"> </span>else</div><div class="gmail_extra"><span style="white-space:pre"> </span>{</div><div class="gmail_extra"><span style="white-space:pre"> </span>memmove(nextFrame->data[0], nextFrame->data[0] + samplesToCopy * sampleSize, (nextFrame->nb_samples - samplesToCopy) * sampleSize);</div><div class="gmail_extra"><span style="white-space:pre"> </span>nextFrame->nb_samples -= samplesToCopy;</div><div class="gmail_extra"><span style="white-space:pre"> </span>}</div><div class="gmail_extra"><span style="white-space:pre"> </span>}</div><div class="gmail_extra"><br></div><div class="gmail_extra">Thanks for your help.</div><div class="gmail_extra"><br></div><div class="gmail_extra">-Kerry</div></div></div></div>