[FFmpeg-user] Remuxing TS to MP4 causes black frame in Apple playback scenarios

[Gyan Doshi]

On 2024-05-02 03:21 am, Sid Sethupathi wrote:
> Hello,
>
> I have a process that remuxes short HLS playlists into an mp4 format. The
> output mp4 in this process has an all black first frame when played back in
> either Safari or Quicktime Player. Everywhere else, the first frame does
> not have this black frame.
>
> The process is an internal tool that uses libav directly so it's hard to
> share the code, but I think I can reproduce via an ffmpeg command line
> invocation against a single TS segment, so I'm hoping to be able to solve
> the problem by understanding how to change the ffmpeg command.
>
> All the sample files can be found in this public Google Drive:
> https://drive.google.com/drive/folders/1L8T5T4l_GUzxRnhjH-ojnT6-44FQLQq1?usp=drive_link
> .
>
> bbb_2.ts is the second segment of Big Buck Bunny (sourced from this test
> stream:
> https://test-streams.mux.dev/x36xhzz/url_0/193039199_mp4_h264_aac_hd_7.m3u8).
> If you open this ts file in Quicktime player or Safari, the first frame is
> NOT black.
>
> bbb_2_remuxed.mp4 has been remuxed using the following command: "ffmpeg -i
> bbb_2.ts -c copy -f mp4 bbb_2_remuxed.mp4". If you open it up in Quicktime
> Player or Safari, the first frame IS black. I think I traced it down to an
> extra entry in the "elst" atom. If you probe this remuxed mp4 with trace
> level debugging on ("ffprobe -v trace bbb_2_remuxed.mp4") there's a short
> duration entry in the edit list of the video track.
>
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd357104bc0] type:'elst' parent:'edts' sz: 40
> 8 40
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd357104bc0] track[0].edit_count = 2
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd357104bc0] duration=15 time=-1 rate=1.000000
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fd357104bc0] duration=10000 time=2999
> rate=1.000000
>
> According to the Quicktime file format docs, the parent of the "elst" atom
> is "edts" and it is optional, so I removed that atom using mp4edit in the
> Bento4 tool kit. This produced "bbb_2_remuxed_no_edts.mp4". If you open
> this file in Quicktime or Safari, the first frame is NOT black.
>
> bbb_2_transcoded.mp4 is a transcoded version using "ffmpeg -i bbb_2.ts
> -video_track_timescale 90K -movflags +faststart bbb_2_transcoded.mp4". If
> you open this file in Quicktime or Safari, the first frame is NOT black. If
> you inspect the atoms in this file, the video "elst" atom is still there,
> but it only has a single entry in the list.
>
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe3ecf06040] type:'elst' parent:'edts' sz: 28
> 8 28
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe3ecf06040] track[0].edit_count = 1
> [mov,mp4,m4a,3gp,3g2,mj2 @ 0x7fe3ecf06040] duration=10017 time=3000
> rate=1.000000
>
> So, ultimately I think the extra entry in the elst edit table is the cause
> of the issue. My question is whether it is possible to not have that happen
> via the ffmpeg/libav invocation, or whether this is a problem with the
> source material?

Reset the pts i.e. first packet of video stream should have pts 0.

Regards,
Gyan