[FFmpeg-trac] #9333(undetermined:new): ffmpeg creating mpeg-dash chunk files too slowly resulting in 404 errors

[FFmpeg]

#9333: ffmpeg creating mpeg-dash chunk files too slowly resulting in 404 errors
-------------------------------------+-------------------------------------
             Reporter:  Danny        |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:
  undetermined                       |  unspecified
             Keywords:               |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 I have a hardware encoder feeding FFmpeg to create a MPEG-DASH Low Latency
 stream. It works well for a while, but after letting FFmpeg run for a
 while and reloading the page there are many 404 errors.

 When that happens, the `dash.js` player tries to fetch the segment file on
 the "live edge" but the file has not been created yet by FFmpeg. For
 example, after running for 20-30 minutes and loading the web page player,
 debug code in the web server shows:

     2021-07-16 16:46:30.64 : GET REQUEST : /data/ott/chunk-
 stream0-00702.m4s
     2021-07-16 16:46:30.67 : NOT FOUND. Latest files on filesystem:
         chunk-stream0-00699.m4s.tmp
         chunk-stream0-00698.m4s
         chunk-stream0-00697.m4s
         chunk-stream0-00696.m4s
         ...

 So you can see the browser requested chunk 702 but the latest on the
 server is (part of) 699.  With 2 second chunks, that is 3-5 seconds of
 content not yet available.

 To analyze, I modified FFmpeg's `dashenc.c` to add a timestamp every time
 a file is opened which displays like:

     [dash @ 0x9b17c0] 21:48:52.935 1626443332.935  : dashenc_io_open() -
 opened /data/ott/chunk-stream0-00060.m4s.tmp

 And loaded the timestamps into Excel.

 Despite a segment duration of 2.000 seconds, the average time between file
 opens is 2.011 seconds.  Over two hours this accumulated to a **45
 second** difference between the calculated live edge and the latest file
 on the server.

 The HW encoder is set to 25 fps and a GOP size of 5.  I've confirmed both
 by analyzing the H.264 NALUs output by the HW encoder.

 Is this a bug in FFmpeg or can I avoid this problem by adjusting the
 settings of either the HW encoder and/or FFmpeg options?


 **REFERENCE**

     FFmpeg: Version 4.4
     Centos 8
     Apache 2.4.37

 *FFmpeg command line (pipe is fed by process reading HW encoder)*

     ffmpeg -re -loglevel verbose -an -f h264 -i pipe:17 -c:v copy \
     -f dash -dash_segment_type mp4 -b:v 1000000 -seg_duration 2.000000 \
     -frag_type duration -frag_duration 0.200000 -target_latency 1 \
     -window_size 10 -extra_window_size 5 -remove_at_exit 1 -streaming 1 \
     -ldash 1 -use_template 1 -use_timeline 0 -write_prft 1 -avioflags
 direct \
     -fflags +nobuffer+flush_packets -format_options movflags=+cmaf \
     -utc_timing_url /web/be/time.php /data/ott/master.mpd

 *Modified `dash_io_open()` from dashenc.c*

     static int
     dashenc_io_open(AVFormatContext *s, AVIOContext **pb, char *filename,
 AVDictionary **options)
     {
         DASHContext *c = s->priv_data;
         int http_base_proto = filename ? ff_is_http_proto(filename) : 0;
         int err = AVERROR_MUXER_NOT_FOUND;
         if (!*pb || !http_base_proto || !c->http_persistent)
         {
                 err = s->io_open(s, pb, filename, AVIO_FLAG_WRITE,
 options);

                 // My Debug
                 {
                     char buf[20], milli[60];
                     struct timeb tp;

                     ftime(&tp); // sec + ms
                     struct tm *tmInfo = localtime(&tp.time);

                     // 2020-05-15 21:15:12.123
                         strftime(buf, sizeof(buf), "%H:%M:%S", tmInfo);
                         snprintf(milli, 59, "%s.%03d %d.%03d ", buf,
 tp.millitm, tp.time, tp.millitm);

                         av_log(s, AV_LOG_INFO, "%s : dashenc_io_open() -
 opened %s\n", milli, filename);
                 }
         }
         return err;
     }
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/9333>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker