<div dir="ltr"><p>I'm new to audio things so I may make some incorrect or silly assumptions, but I'm hoping someone can help!<br><br>I'm making a C++ project that receives the audio data of a file that is streamed in chunks. It will demux it, then decode and further process it. The audio can be any audio file format, so I expect the beginning of the data to be the header bytes and then encoded audio data to follow. But I do not have all the audio data at the start, and I want to start processing it while I am still receiving more!<br><br>[Code below] I'm making a custom AVIOContext and using it to initialise an AVFormatContext. I open the context using avformat_open_input() and make sure the first chunk of data is available to my custom read function in the AVIOContext. Then I repeatedly call av_read_frame() to extract the encoded audio packets.<br><br>However, because the audio is coming as a stream, I don't have all the audio available at once. So when I call av_read_frame() ffmpeg will read through my data with my read function until it gets to the end of available data, and then it treats it like an end of file (even though I return AVERROR(EAGAIN)). Then I receive more audio data to put at the end of my previous data, and I call av_read_frame() again to process the new data. But this time when I call it, it immediately fails with EOF because the AVFormatContext thinks it reached the end of the file the last time (even though I returned EAGAIN). The only workaround I have is to stop calling av_read_frame() when ffmpeg reads the last bit of my buffer, so it hopefully doesn't try to read more, until I receive more. But this means there will be a few packets left in the demuxer that I don't process until I get the next chunk, and I really want to process all available data as soon as possible (it should arrive in framed-aligned chunks).<br><br>Is this actually possible with an AVFormatContext? Should my read function return something special? Do I need to set a special value in the AVFormatContext to make it work when data isn't immediately available? Or can I reset the AVFormatContext back to a good non-EOF state somehow? Or should I use my workaround and try to never let it read to the end of the available data until the stream has finished?<br><br>All help is appreciated. Thank you<br><br>Here is my code (error handling removed):<br></p><p>-------------------------------------------</p><div style="font-family:"Droid Sans Mono","monospace",monospace;font-size:14px;line-height:19px;white-space:pre-wrap"><div style="color:rgb(59,59,59)"><span style="color:rgb(0,0,255)">int</span> <span style="color:rgb(121,94,38)">read_packet</span>(<span style="color:rgb(0,0,255)">void*</span> <span style="color:rgb(0,16,128)">opaque</span>, <span style="color:rgb(0,0,255)">uint8_t*</span> <span style="color:rgb(0,16,128)">buffer</span>, <span style="color:rgb(0,0,255)">int</span> <span style="color:rgb(0,16,128)">bufferSize</span>) {</div><div style="color:rgb(59,59,59)"> StreamedDataAccessor<span style="color:rgb(0,0,0)">*</span> dataAccessor <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(0,0,255)">static_cast</span><span style="color:rgb(0,0,0)"><</span>StreamedDataAccessor<span style="color:rgb(0,0,0)">*></span>(opaque);</div><div style="color:rgb(59,59,59)"> </div><div style="color:rgb(59,59,59)"> <span style="color:rgb(0,0,255)">size_t</span> bytesToRead <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(38,127,153)">std</span>::<span style="color:rgb(121,94,38)">min</span>((<span style="color:rgb(0,0,255)">size_t</span>)bufferSize, <span style="color:rgb(0,16,128)">dataAccessor-></span><span style="color:rgb(121,94,38)">remainingBytes</span>());</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(175,0,219)">if</span> (<span style="color:rgb(0,0,0)">!</span>bytesToRead) {</div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // Need to wait until we get the next chunk of data</span><br></div><div style="color:rgb(59,59,59)"> <span style="color:rgb(175,0,219)">return</span> <span style="color:rgb(121,94,38)">AVERROR</span>(EAGAIN);</div><div style="color:rgb(59,59,59)"> }</div><font color="#3b3b3b"><br></font><div style=""><font color="#3b3b3b"> </font><span style="color:rgb(0,0,255)">size_t</span><font color="#3b3b3b"> bytesRead </font><span style="color:rgb(0,0,0)">=</span><font color="#3b3b3b"> </font><span style="color:rgb(0,16,128)">dataAccessor</span><font color="#3b3b3b">-></font><span style="color:rgb(121,94,38)">copyData</span><font color="#3b3b3b">(buffer, bytesToRead);</font></div><div style="color:rgb(59,59,59)"> <span style="color:rgb(175,0,219)">return</span> bytesRead;</div><div style="color:rgb(59,59,59)">}</div><font color="#3b3b3b"><br></font><div style="color:rgb(59,59,59)"><span style="color:rgb(0,0,255)">bool</span> <span style="color:rgb(121,94,38)">processHeaderPacket</span>(<span style="color:rgb(0,0,255)">const</span> <span style="color:rgb(38,127,153)">Data</span> <span style="color:rgb(0,16,128)">headerPacket</span>) {</div><div style="color:rgb(59,59,59)"> dataAccessor <span style="color:rgb(0,0,0)">=</span> StreamedDataAccessor{</div><div style="color:rgb(59,59,59)"> .header <span style="color:rgb(0,0,0)">=</span> headerPacket,</div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // Set up other internal stuff</span></div><div style="color:rgb(59,59,59)"> };
</div><font color="#3b3b3b"><br></font><div style="color:rgb(59,59,59)"> <span style="color:rgb(0,0,255)">size_t</span> bufferSize <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(9,134,88)">4096</span>;</div><div style="color:rgb(59,59,59)"> contextBuffer <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(121,94,38)">allocateEmptyBuffer</span>(bufferSize);</div><div style="color:rgb(59,59,59)"> IOContext <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(121,94,38)">avio_alloc_context</span>(<span style="color:rgb(0,16,128)">contextBuffer</span>.<span style="color:rgb(0,16,128)">ptr</span>,</div><div style="color:rgb(59,59,59)"> bufferSize,</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(9,134,88)">0</span>,</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(0,0,0)">&</span>(dataAccessor),</div><div style="color:rgb(59,59,59)"> read_packet,</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(9,134,88)">0</span>,</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(9,134,88)">0</span>);</div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // no seek_function provided</span></div><font color="#3b3b3b"><br></font><div style="color:rgb(59,59,59)"> AVFormatContext<span style="color:rgb(0,0,0)">*</span> inFormatContext <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(121,94,38)">avformat_alloc_context</span>();</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(0,16,128)">inFormatContext</span>-><span style="color:rgb(0,16,128)">pb</span> <span style="color:rgb(0,0,0)">=</span> IOContext;</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(0,16,128)">inFormatContext</span>-><span style="color:rgb(0,16,128)">flags</span> <span style="color:rgb(0,0,0)">=</span> AVFMT_FLAG_CUSTOM_IO;</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(0,0,255)">int</span> ret <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(121,94,38)">avformat_open_input</span>(<span style="color:rgb(0,0,0)">&</span>inFormatContext, <span style="color:rgb(163,21,21)">""</span>, <span style="color:rgb(0,0,255)">nullptr</span>, <span style="color:rgb(0,0,255)">nullptr</span>);</div><font color="#3b3b3b"><br></font><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // Error handling</span></div><div style="color:rgb(59,59,59)"> <span style="color:rgb(175,0,219)">if</span> (ret <span style="color:rgb(0,0,0)">==</span> ...) {}</div><font color="#3b3b3b"><br></font><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // Probably only necessary for mpgs</span></div><div style="color:rgb(59,59,59)"> <span style="color:rgb(175,0,219)">if</span> (<span style="color:rgb(121,94,38)">avformat_find_stream_info</span>(inFormatContext, <span style="color:rgb(0,0,255)">nullptr</span>) <span style="color:rgb(0,0,0)"><</span> <span style="color:rgb(9,134,88)">0</span>) {</div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // ERROR could not find stream info</span></div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // ...</span></div><div style="color:rgb(59,59,59)"> }</div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // Now we access stream info, codec info, etc</span></div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // ...</span></div><div style="color:rgb(59,59,59)">}</div><font color="#3b3b3b"><br><br></font><div style="color:rgb(59,59,59)">inputPacket <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(121,94,38)">av_packet_alloc</span>();</div><font color="#3b3b3b"><br></font><div style="color:rgb(59,59,59)"><span style="color:rgb(0,0,255)">void</span> <span style="color:rgb(121,94,38)">processData</span>(<span style="color:rgb(38,127,153)">AudioData</span> <span style="color:rgb(0,16,128)">data</span>) {</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(0,16,128)">dataAccessor</span>.<span style="color:rgb(121,94,38)">appendData</span>(data);</div><div style="color:rgb(59,59,59)"> <span style="color:rgb(0,0,255)">int</span> ret <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(9,134,88)">0</span>; </div><div style="color:rgb(59,59,59)"> <span style="color:rgb(175,0,219)">while</span> ((ret <span style="color:rgb(0,0,0)">=</span> <span style="color:rgb(121,94,38)">av_read_frame</span>(inFormatContext, inputPacket)) <span style="color:rgb(0,0,0)">>=</span> <span style="color:rgb(9,134,88)">0</span>) {</div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // process inputPacket further</span></div><div style="color:rgb(59,59,59)"> }</div><div style="color:rgb(59,59,59)"><br></div><div style=""><div style="line-height:19px"><div style="color:rgb(59,59,59)"><font color="#3b3b3b"> </font><span style="color:rgb(175,0,219)">if</span><font color="#3b3b3b"> (</font><span style="color:rgb(0,16,128)">ret</span><font color="#3b3b3b"> </font><span style="color:rgb(0,0,0)">==</span><font color="#3b3b3b"> </font><span style="color:rgb(0,0,255)">AVERROR</span><font color="#3b3b3b">(</font><span style="color:rgb(0,0,255)">EAGAIN</span><font color="#3b3b3b">)) {</font><br></div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // Hits here the first time, once ffmpeg has read all available data</span></div><div style="color:rgb(59,59,59)"> } <span style="color:rgb(175,0,219)">else</span> <span style="color:rgb(175,0,219)">if</span> (<span style="color:rgb(0,16,128)">ret</span> <span style="color:rgb(0,0,0)">==</span> <span style="color:rgb(0,0,255)">AVERROR_EOF</span>) {<br></div><div style="color:rgb(59,59,59)"><div><span style="color:rgb(0,128,0)"> // Hits here the second time, after we receive more data. av_read_frame immediately returns</span></div></div><div style="color:rgb(59,59,59)"> } <span style="color:rgb(175,0,219)">else</span> {<br></div><div style="color:rgb(59,59,59)"><span style="color:rgb(0,128,0)"> // other error handling here</span></div><div style="color:rgb(59,59,59)"> }</div></div></div><div style="color:rgb(59,59,59)">}</div></div><p><br></p></div>