[FFmpeg-trac] #10380(ffmpeg:new): H264/AAC MP4 video first frame doesn't report a zero mediaTime using HTML requestVideoFrameCallback API

[FFmpeg]

#10380: H264/AAC MP4 video first frame doesn't report a zero mediaTime using HTML
requestVideoFrameCallback API
-------------------------------------+-------------------------------------
             Reporter:  Daniel       |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:  ffmpeg       |                  Version:
             Keywords:  AAC mp4      |  unspecified
  timestamp                          |               Blocked By:
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 **Version**: 6.0-essentials_build-www.gyan.dev or
 N-109421-g9adf02247c-20221216

 **Platform**: Windows 10 (Browser: Chrome 113.0.5672.92)

 **Summary:** Mp4 files generated using ffmpeg aren’t reporting a mediaTime
 of 0 via requestVideoFrameCallback for the first frame. This behavior is
 not observed with files encoded as webm or via another encoder such as
 MainConcept.

 **Description:**
 When incrementing frames with video 1, progress is not visible until the
 third button press in the demo provided here: https://randomstuffdr.s3.eu-
 west-1.amazonaws.com/TEST-v3.HTML

 The mediaTime issue may be due to the aac encoder in ffmpeg, as the
 problem does not occur when using aac_mf.

 This might be related to how Chromium implements mediaTime, using the
 audio clock as the time source that backs video.currentTime, while the
 mediaTime is directly populated by the presentationTimestamp of the frame,
 as noted here: https://web.dev/requestvideoframecallback-rvfc/

 **Actual results:**

 No video progress is observed until the third button press.

 **Expected results:**

 Video progress should be visible from the first frame.

 **Additional information:**

 Attempts to resolve the issue with ffmpeg (disabling b-frames, use
 baseline profile and avoid_negative_ts) have not been successful.

 **Question:**

 Is there a way to generate a file in a similar manner to MainConcept or
 ffmpeg when using acc_mf?


 **Commands used to generate files**
 Video 1 – (Video: H264 /Audio: AAC / Wrapper: MP4 / Transcoder: FFMPEG)
 .\ffmpeg.exe -i '720p50 Flash and Beep.mxf' -map 0:0 -map 0:1
 -map_metadata -1 -c:v
 libx264 -profile:v baseline -level:v 2.0 -pix_fmt yuv420p -crf 28 -b:v
 118k -maxrate 118k -bufsize 236k -vf scale="480:-1" -preset veryfast -c:a
 aac "..\low-res\720p50FlashandBeep_V_A.mp4"

 Video 2 – (Video: H264 /Audio: AAC / Wrapper: MP4 / Transcoder: FFMPEG)
 .\ffmpeg.exe -i '720p50 Flash and Beep.mxf' -map 0:0 -map 0:1
 -map_metadata -1 -c:v
 libx264 -profile:v baseline -level:v 2.0 -pix_fmt yuv420p -crf 28 -b:v
 118k -maxrate 118k -bufsize 236k -vf scale="480:-1" -preset veryfast -c:a
 aac_mf "..\low-res\720p50FlashandBeep_V_A_mf.mp4"

 Video 3 – (Video: H264 /Audio: No Audio / Wrapper: MP4 / Transcoder:
 FFMPEG) .\ffmpeg.exe -i '720p50 Flash and Beep.mxf' -map 0:0 -map 0:1
 -map_metadata -1 -c:v
 libx264 -profile:v baseline -level:v 2.0 -pix_fmt yuv420p -crf 28 -b:v
 118k -maxrate 118k -bufsize 236k -vf scale="480:-1" -preset veryfast -an
 "..\low-res\720p50FlashandBeep_V.mp4"

 Video 4 – (Video: vp8 /Audio: ogg / Wrapper: webm / Transcoder: FFMPEG)
 .\ffmpeg.exe -i '720p50 Flash and Beep.mxf' -map 0:0 -map 0:1
 -map_metadata -1 -c:v vp8 -crf 28 -b:v 118k -maxrate 118k -bufsize 236k
 -vf scale="480:-1" -preset veryfast "..\low-
 res\720p50FlashandBeep_V_A.webm"

 Video 5 – (Video: H264 /Audio: AAC / Wrapper: MP4 / Transcoder:
 MainConcept)
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10380>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker