<div dir="ltr"><div>As I have made no progress on this problem, I have made a small repo with a testcase and two sample transport streams. </div><div><br></div><div><a href="https://github.com/taxfromdk/demux_delay">https://github.com/taxfromdk/demux_delay</a></div><div><a href="https://github.com/taxfromdk/demux_delay/blob/master/main.cpp">https://github.com/taxfromdk/demux_delay/blob/master/main.cpp</a><br></div><div><br></div><div>I hope that it will help me get some feedback to progress from.<br></div><div><br></div><div><div>The problem is that when my stream contains customdata, that data triggers something in my (mpegts) demuxer that keeps the demuxer from returning AVPackets for more than 2.8MB of the stream. That delays my playback with almost 5 seconds which is quite annoying.</div><div><br></div><div>When there is no customdata is in the stream the packages are received at much more regular intervals.</div><div><br></div></div><div>I am on "avformat_version()" version 3808612</div><div><br></div><div>The output I get here is (streamindex 2 is the customdata):</div><b><br>make test<br><br></b><div><b>Testing with Plain<br>./main plain.ts<br>plain.ts read into 1382 chunks<br>We are using libavformat version: 3808612<br>Package returned after 34816 bytes read. It has StreamIndex: 0 and is 24681 bytes long.<br>Package returned after 38912 bytes read. It has StreamIndex: 0 and is 6510 bytes long.<br>Package returned after 45056 bytes read. It has StreamIndex: 0 and is 3350 bytes long.<br>Package returned after 49152 bytes read. It has StreamIndex: 0 and is 4758 bytes long.<br>Package returned after 59392 bytes read. It has StreamIndex: 0 and is 3394 bytes long.<br>Package returned after 69632 bytes read. It has StreamIndex: 1 and is 306 bytes long.<br>First video package returned after having read 69632 bytes.<br><br></b><div><b>Testing with Customdata<br>customdata.ts read into 1418 chunks<br>We are using libavformat version: 3808612<br>Package returned after 2824192 bytes read. It has StreamIndex: 2 and is 36 bytes long.<br>Package returned after 2824192 bytes read. It has StreamIndex: 2 and is 36 bytes long.<br>Package returned after 2824192 bytes read. It has StreamIndex: 2 and is 36 bytes long.<br>Package returned after 2824192 bytes read. It has StreamIndex: 1 and is 24681 bytes long.<br>First video package returned after having read 2824192 bytes.</b><div><br></div><div>The test case shows that the first video AVPacket is returned after 69632 bytes when there is no customdata in the stream. With customdata in the stream the first video AVPacket is returned after 2.8MB.</div><div><br></div><div>I have tried setting variables like <b>max_analyze_duration</b> on my <b>AVFormatContext</b>, but with no luck yet.</div><div><br></div><div>The ffprobe output of customdata.ts is </div><div><br></div><b>ffprobe -hide_banner customdata.ts<br>Input #0, mpegts, from 'customdata.ts':<br> Duration: 00:00:52.28, start: 0.062000, bitrate: 2220 kb/s<br> Program 1 <br> Metadata:<br> service_name : Service01<br> service_provider: FFmpeg<br> Stream #0:0[0x46]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 48000 Hz, stereo, fltp, 93 kb/s<br> Stream #0:1[0x45]: Video: hevc (Main) (HEVC / 0x43564548), yuv420p(tv), 1280x720 [SAR 1:1 DAR 16:9], 30 fps, 30 tbr, 90k tbn, 30 tbc<br> Stream #0:2[0x47]: Data: bin_data ([6][0][0][0] / 0x0006)<br>Unsupported codec with id 100359 for input stream 2</b><div><br></div><div>Any hints that can help me deal with this issue will be highly appreciated.</div><div><br></div><div>Kind regards</div><div><br></div><div>Jesper</div></div></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">Den søn. 24. maj 2020 kl. 00.29 skrev Jesper Taxbøl <<a href="mailto:jesper@taxboel.dk">jesper@taxboel.dk</a>>:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="auto"><div dir="ltr"><div>Hi</div><div dir="auto"><br></div><div>I am working on a streaming video application with a client and a server. </div><div><br></div><div>My transport stream contains HEVC, AAC and a customdata track. </div><div><br></div><div>In the server I am using these settings on the data stream.</div><div><br></div><div>codec_type = AVMEDIA_TYPE_DATA;<br><div>codec_id = AV_CODEC_ID_BIN_DATA; </div><div></div></div><div><br></div><div>The data is a small byte chunk sent for every frame with some parameters relating to the video. Fixed size 38bytes. Video, audio and data pass through the demuxer unharmed.<br></div><div><br></div><div><div><div>But, I am experiencing that my AVFormatContext returns the first ~20 packages within rrecieving the first 18000 bytes. Then it stalls for almost 2.8MB before returning the following packages.</div></div></div><div dir="auto"><br></div><div dir="auto">It must be noted that I use an AVIOContext to feed the data into the AVFormatContext.</div><div dir="auto"><br></div><div dir="auto">When debugging I have tried some changes and found that;</div><div><br></div><div>If I dont inject the actual data packets there is much less delay. So I suspect my problem has something to do with me triggering some analysis of the stream with my data packets. </div><div><br></div><div>Is there a way to tell the AVFormatContext what the contents of the stream is, so I can get data from the stream with as little delay as posible?</div><div><br></div><div>I have added my test setup below, where the socket is replaced with a chunkwise data feeder.</div><div><br></div><div>kind regards</div><div><br></div><div>Jesper</div><div><br></div><div><br></div><div><div style="color:rgb(212,212,212);background-color:rgb(30,30,30);font-family:"Droid Sans Mono",monospace,monospace,"Droid Sans Fallback";font-size:14px;line-height:19px;white-space:pre-wrap"><div><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><stdio.h></span></div><div><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><queue></span></div><br><br><div><span style="color:rgb(197,134,192)">using</span> <span style="color:rgb(86,156,214)">namespace</span> <span style="color:rgb(78,201,176)">std</span>;</div><br><div><span style="color:rgb(197,134,192)">#define</span><span style="color:rgb(86,156,214)"> __STDC_CONSTANT_MACROS</span></div><br><div><span style="color:rgb(86,156,214)">extern</span> <span style="color:rgb(206,145,120)">"C"</span> {</div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libavutil/avassert.h></span></div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libavutil/channel_layout.h></span></div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libavutil/opt.h></span></div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libavutil/mathematics.h></span></div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libavutil/timestamp.h></span></div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libavformat/avformat.h></span></div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libswscale/swscale.h></span></div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libswresample/swresample.h></span></div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#include</span><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(206,145,120)"><libavutil/file.h></span></div><div>}</div><br><div><span style="color:rgb(86,156,214)">struct</span> <span style="color:rgb(78,201,176)">Packet</span>{</div><div> <span style="color:rgb(86,156,214)">uint8_t</span>* data;</div><div> <span style="color:rgb(86,156,214)">uint64_t</span> len;</div><div>};</div><br><div>queue<Packet*> Q;</div><div><span style="color:rgb(86,156,214)">static</span> <span style="color:rgb(86,156,214)">int</span> x = <span style="color:rgb(181,206,168)">0</span>;</div><div><span style="color:rgb(86,156,214)">uint64_t</span> bc = <span style="color:rgb(181,206,168)">0</span>;</div><div><span style="color:rgb(86,156,214)">static</span> <span style="color:rgb(86,156,214)">int</span> <span style="color:rgb(220,220,170)">read_packet</span>(<span style="color:rgb(86,156,214)">void</span> <span style="color:rgb(86,156,214)">*</span><span style="color:rgb(156,220,254)">opaque</span>, <span style="color:rgb(86,156,214)">uint8_t</span> <span style="color:rgb(86,156,214)">*</span><span style="color:rgb(156,220,254)">buf</span>, <span style="color:rgb(86,156,214)">int</span> <span style="color:rgb(156,220,254)">buf_size</span>)</div><div>{</div><div> <span style="color:rgb(86,156,214)">int</span> r = <span style="color:rgb(181,206,168)">0</span>;</div><br><div><span style="color:rgb(106,153,85)"> //printf("x:%d %llu\r\n", x++, bc);</span></div><div> <span style="color:rgb(197,134,192)">if</span>(x == <span style="color:rgb(181,206,168)">35</span>)</div><div> {</div><div> <span style="color:rgb(86,156,214)">int</span> x;</div><div> x = <span style="color:rgb(181,206,168)">2</span>;</div><div> }</div><div> <span style="color:rgb(197,134,192)">if</span>(<span style="color:rgb(156,220,254)">Q</span>.<span style="color:rgb(220,220,170)">size</span>() > <span style="color:rgb(181,206,168)">0</span>)</div><div> {</div><div> Packet* p = <span style="color:rgb(156,220,254)">Q</span>.<span style="color:rgb(220,220,170)">front</span>();</div><div> <span style="color:rgb(197,134,192)">if</span>(<span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">len</span> > buf_size)</div><div> {</div><div><span style="color:rgb(106,153,85)"> //printf("Cut\r\n");</span></div><div> <span style="color:rgb(220,220,170)">memcpy</span>(buf, <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">data</span>, buf_size);</div><div> <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">len</span> -= buf_size;</div><div> <span style="color:rgb(86,156,214)">uint8_t</span>* new_data = (<span style="color:rgb(86,156,214)">uint8_t</span>*)<span style="color:rgb(220,220,170)">malloc</span>(<span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">len</span>);</div><div> <span style="color:rgb(220,220,170)">memcpy</span>(new_data, &<span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">data</span>[buf_size], <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">len</span>);</div><div> <span style="color:rgb(220,220,170)">free</span>(<span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">data</span>);</div><div> <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">data</span> = new_data;</div><div> r = buf_size; </div><div> }</div><div> <span style="color:rgb(197,134,192)">else</span></div><div> {</div><div><span style="color:rgb(106,153,85)"> //printf("Full\r\n");</span></div><div> <span style="color:rgb(220,220,170)">memcpy</span>(buf, <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">data</span>, <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">len</span>);</div><div> <span style="color:rgb(220,220,170)">free</span>(<span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">data</span>);</div><div> r = <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">len</span>;</div><div> <span style="color:rgb(156,220,254)">Q</span>.<span style="color:rgb(220,220,170)">pop</span>();</div><div> <span style="color:rgb(220,220,170)">free</span>(p);</div><div> }</div><br><div> </div><div> }</div><div> bc += r;</div><div> <span style="color:rgb(197,134,192)">return</span> r;</div><div>}</div><br><br><br><div><span style="color:rgb(86,156,214)">int</span> <span style="color:rgb(220,220,170)">main</span>(<span style="color:rgb(86,156,214)">int</span> <span style="color:rgb(156,220,254)">argc</span>, <span style="color:rgb(86,156,214)">char*</span> <span style="color:rgb(156,220,254)">argv</span>[])</div><div>{</div><div> <span style="color:rgb(220,220,170)">printf</span>(<span style="color:rgb(206,145,120)">"Hello world...</span><span style="color:rgb(215,186,125)">\r\n</span><span style="color:rgb(206,145,120)">"</span>);</div><br><div> <span style="color:rgb(86,156,214)">static</span> AVFormatContext *fmt_ctx = <span style="color:rgb(86,156,214)">NULL</span>;</div><div><span style="color:rgb(106,153,85)"> //fmt_ctx->flags |= AVFMT_FLA</span></div><br><div><span style="color:rgb(106,153,85)"> //Read file data into queue packets</span></div><div> FILE* f = <span style="color:rgb(220,220,170)">fopen</span>(<span style="color:rgb(206,145,120)">"customdata.ts"</span>, <span style="color:rgb(206,145,120)">"rb"</span>);</div><div><span style="color:rgb(86,156,214)"> </span><span style="color:rgb(197,134,192)">#define</span><span style="color:rgb(86,156,214)"> BUF_SIZE </span><span style="color:rgb(181,206,168)">1000</span></div><div> <span style="color:rgb(86,156,214)">uint8_t</span> <span style="color:rgb(156,220,254)">b</span>[BUF_SIZE];</div><div> <span style="color:rgb(197,134,192)">while</span>(<span style="color:rgb(181,206,168)">1</span>)</div><div> {</div><div> <span style="color:rgb(86,156,214)">int</span> l = <span style="color:rgb(220,220,170)">fread</span>(b, <span style="color:rgb(181,206,168)">1</span>, BUF_SIZE, f);</div><div> <span style="color:rgb(197,134,192)">if</span>(l <= <span style="color:rgb(181,206,168)">0</span>)</div><div> {</div><div> <span style="color:rgb(197,134,192)">break</span>;</div><div> }</div><div> Packet* p = (Packet*)<span style="color:rgb(220,220,170)">malloc</span>(<span style="color:rgb(86,156,214)">sizeof</span>(Packet));</div><div> <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">data</span> = (<span style="color:rgb(86,156,214)">uint8_t</span>*)<span style="color:rgb(220,220,170)">malloc</span>(l);</div><div> <span style="color:rgb(220,220,170)">memcpy</span>((<span style="color:rgb(86,156,214)">void</span>*)<span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">data</span>, (<span style="color:rgb(86,156,214)">void</span>*)b, l);</div><div> <span style="color:rgb(156,220,254)">p</span>-><span style="color:rgb(156,220,254)">len</span> = l;</div><div> <span style="color:rgb(156,220,254)">Q</span>.<span style="color:rgb(220,220,170)">push</span>(p);</div><div> }</div><div> <span style="color:rgb(220,220,170)">fclose</span>(f);</div><div> <span style="color:rgb(220,220,170)">printf</span>(<span style="color:rgb(206,145,120)">"File read into %d chunks</span><span style="color:rgb(215,186,125)">\r\n</span><span style="color:rgb(206,145,120)">"</span>, <span style="color:rgb(156,220,254)">Q</span>.<span style="color:rgb(220,220,170)">size</span>());</div><br><div> </div><br><div> <span style="color:rgb(197,134,192)">if</span> (!(fmt_ctx = <span style="color:rgb(220,220,170)">avformat_alloc_context</span>())) {</div><div> <span style="color:rgb(220,220,170)">fprintf</span>(stderr, <span style="color:rgb(206,145,120)">"Could not allocate fmt_ctx</span><span style="color:rgb(215,186,125)">\n</span><span style="color:rgb(206,145,120)">"</span>);</div><div> <span style="color:rgb(220,220,170)">exit</span>(<span style="color:rgb(181,206,168)">1</span>);</div><div> }</div><br><div> <span style="color:rgb(86,156,214)">size_t</span> avio_ctx_buffer_size = <span style="color:rgb(181,206,168)">4096</span>;</div><br><div> <span style="color:rgb(86,156,214)">uint8_t</span>* avio_ctx_buffer = (<span style="color:rgb(86,156,214)">uint8_t</span>*)<span style="color:rgb(220,220,170)">av_malloc</span>(avio_ctx_buffer_size);</div><div> <span style="color:rgb(197,134,192)">if</span> (!avio_ctx_buffer) {</div><div> <span style="color:rgb(220,220,170)">printf</span>(<span style="color:rgb(206,145,120)">"error allocating avio_ctx_buffer</span><span style="color:rgb(215,186,125)">\r\n</span><span style="color:rgb(206,145,120)">"</span>);</div><div> <span style="color:rgb(220,220,170)">exit</span>(<span style="color:rgb(181,206,168)">0</span>);</div><div> }</div><br><div> AVIOContext* avio_ctx = <span style="color:rgb(220,220,170)">avio_alloc_context</span>(avio_ctx_buffer, avio_ctx_buffer_size, <span style="color:rgb(181,206,168)">0</span>, <span style="color:rgb(86,156,214)">NULL</span>, &read_packet, <span style="color:rgb(86,156,214)">NULL</span>, <span style="color:rgb(86,156,214)">NULL</span>);</div><br><div> <span style="color:rgb(197,134,192)">if</span> (!avio_ctx) {</div><div> <span style="color:rgb(220,220,170)">printf</span>(<span style="color:rgb(206,145,120)">"error allocating avio_ctx</span><span style="color:rgb(215,186,125)">\r\n</span><span style="color:rgb(206,145,120)">"</span>);</div><div> <span style="color:rgb(220,220,170)">exit</span>(<span style="color:rgb(181,206,168)">0</span>);</div><div> }</div><div> </div><div> <span style="color:rgb(156,220,254)">fmt_ctx</span>-><span style="color:rgb(156,220,254)">pb</span> = avio_ctx;</div><br><br><br><br><div> AVInputFormat *inputFormat = <span style="color:rgb(220,220,170)">av_find_input_format</span>(<span style="color:rgb(206,145,120)">"mpegts"</span>);</div><div> AVDictionary *inOptions = <span style="color:rgb(86,156,214)">NULL</span>;</div><div> <span style="color:rgb(220,220,170)">av_dict_set</span>(&inOptions, <span style="color:rgb(206,145,120)">"test"</span>,<span style="color:rgb(206,145,120)">"test"</span>, <span style="color:rgb(181,206,168)">0</span>);</div><div> <span style="color:rgb(197,134,192)">if</span> (<span style="color:rgb(220,220,170)">avformat_open_input</span>(&fmt_ctx, <span style="color:rgb(86,156,214)">NULL</span>, inputFormat, &inOptions) < <span style="color:rgb(181,206,168)">0</span>)</div><div> {</div><div> <span style="color:rgb(220,220,170)">fprintf</span>(stderr, <span style="color:rgb(206,145,120)">"Could not open source file </span><span style="color:rgb(215,186,125)">\r\n</span><span style="color:rgb(206,145,120)">"</span>);</div><div> <span style="color:rgb(220,220,170)">exit</span>(<span style="color:rgb(181,206,168)">1</span>);</div><div> }</div><br><div> AVPacket pkt;</div><div> <span style="color:rgb(220,220,170)">av_init_packet</span>(&pkt);</div><div> <span style="color:rgb(156,220,254)">pkt</span>.<span style="color:rgb(156,220,254)">data</span> = <span style="color:rgb(86,156,214)">NULL</span>;</div><div> <span style="color:rgb(156,220,254)">pkt</span>.<span style="color:rgb(156,220,254)">size</span> = <span style="color:rgb(181,206,168)">0</span>;</div><div> <span style="color:rgb(86,156,214)">int</span> framecounter = <span style="color:rgb(181,206,168)">0</span>;</div><div> <span style="color:rgb(197,134,192)">while</span>(<span style="color:rgb(220,220,170)">av_read_frame</span>(fmt_ctx, &pkt) >= <span style="color:rgb(181,206,168)">0</span>)</div><div> {</div><div> <span style="color:rgb(220,220,170)">printf</span>(<span style="color:rgb(206,145,120)">"index %d %llu %d</span><span style="color:rgb(215,186,125)">\r\n</span><span style="color:rgb(206,145,120)">"</span>, framecounter, bc, <span style="color:rgb(156,220,254)">pkt</span>.<span style="color:rgb(156,220,254)">stream_index</span>);</div><div> <span style="color:rgb(220,220,170)">av_free_packet</span>(&pkt); </div><div> <span style="color:rgb(197,134,192)">if</span>(framecounter == <span style="color:rgb(181,206,168)">22</span>)</div><div> {</div><div> <span style="color:rgb(220,220,170)">printf</span>(<span style="color:rgb(206,145,120)">"now somethin funny start</span><span style="color:rgb(215,186,125)">\r\n</span><span style="color:rgb(206,145,120)">"</span>);</div><div> }</div><div> <span style="color:rgb(197,134,192)">if</span>(framecounter++ > <span style="color:rgb(181,206,168)">100</span>)</div><div> {</div><div> <span style="color:rgb(197,134,192)">break</span>;</div><div> }</div><div> }</div><br><div> <span style="color:rgb(197,134,192)">return</span> <span style="color:rgb(181,206,168)">0</span>;</div><div>}</div></div></div></div></div>
</blockquote></div><br clear="all"><div><br></div>-- <br><div dir="ltr" class="gmail_signature">Jesper Taxbøl<br>+45 61627501<br><br></div>