[FFmpeg-trac] #10369(undetermined:new): Front and end padding not trimmed from m4a (dash) causing gapless playback to fail

[FFmpeg]

#10369: Front and end padding not trimmed from m4a (dash) causing gapless playback
to fail
-------------------------------------+-------------------------------------
             Reporter:  Patrick Kan  |                     Type:  defect
               Status:  new          |                 Priority:  normal
            Component:               |                  Version:
  undetermined                       |  unspecified
             Keywords:  m4a aac      |               Blocked By:
  dash gapless                       |
             Blocking:               |  Reproduced by developer:  0
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------
 When using FFmpeg to concat the m4a samples from
 [https://www2.iis.fraunhofer.de/AAC/gapless.html], the output is one that
 plays seamlessly. These samples are of ftyp->major_brand 'mp42' (according
 to ffprobe).

 On the other hand, m4a streams from YouTube / YouTube Music are of
 ftyp->major brand 'dash'. When concatenating streams downloaded from YT,
 the samples corresponding to front and end padding are not discarded. The
 output is therefore one that contains gaps at points where streams meet.

 On YouTube Music in particular, provided you have a Premium subscription,
 you can enjoy gapless playback of performances spanning multiple tracks.
 An example is Pink Floyd's The Wall album.

 My limited understanding is that for MP4 containers, the edts/elst boxes
 provide information about the front padding (is it called encoder delay?)
 as well as the track duration, which can be used to determine the end
 padding. I have verified that these are contained in the YT streams, but
 FFmpeg doesn't seem to use them as it does with "regular" m4a files.

 My use case is to actually pass the URLs of the YT / YT Music streams to
 MusicPlayerDaemon, which will then utilize FFmpeg to decode them. But I
 suppose as long as FFmpeg deals with edts/elst for these streams, and
 remove the front and end padding during decoding, then it shouldn't matter
 what player I use in the end.

 The following illustrates the steps I took in diagnosing the issue
 (emphasis: my knowledge is limited):

 1. Download streams with youtube-dl for closer inspection:

 {{{
 // Album: Pink Floyd - The Wall
 // Track 2
 # youtube-dl --fixup never -v -f m4a
 "https://music.youtube.com/watch?v=Ciai1aZ_odg"

 // Track 3
 # youtube-dl --fixup never -v -f m4a
 "https://music.youtube.com/watch?v=-cfJqYtmmqA"

 // Above commands create the following files for track 2 and 3
 respectively:
 // 'The Thin Ice-Ciai1aZ_odg.m4a'
 // 'Another Brick In The Wall (Part 1)--cfJqYtmmqA.m4a'
 }}}


 2. Verify that edts/elst exist in stream:

 {{{
 # mediainfo --DETAILS=1 The The\ Thin\ Ice-Ciai1aZ_odg.m4a
 }}}

 (Output in attached mediainfo.txt)

 3. Concat the two m4a files created in step 1:

 {{{
 # ffmpeg -v 9 -loglevel 99 -f concat -safe 0 -report -i concat.txt
 concat.m4a

 // concat.txt:
 file 'The Thin Ice-Ciai1aZ_odg.m4a'
 file 'Another Brick In The Wall (Part 1)--cfJqYtmmqA.m4a'
 }}}

 (Output in attached ffmpeg-20230516-212923.log)

 The logs do not mention FFmpeg skipping or discarding any samples. This is
 different from concatenating "regular" aac files, which I did as follows
 for testing purpose:

 {{{
 # wget https://www2.iis.fraunhofer.de/AAC/gapless-sweep_part1_iis.m4a
 # wget https://www2.iis.fraunhofer.de/AAC/gapless-sweep_part2_iis.m4a
 # ffmpeg -v 9 -loglevel 99 -f concat -safe 0 -report -i concat-regular.txt
 concat-regular.m4a

 // concat-regular.txt
 file 'gapless-sweep_part1_iis.m4a'
 file 'gapless-sweep_part2_iis.m4a'
 }}}

 (Output in attached ffmpeg-20230517-004305.log)

 From logs, FFmpeg did skip some samples due to side data (which I presume
 refers to edts / elst?):
 {{{
 # cat ffmpeg-20230517-004305.log | grep skip
 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x558bdd922d00] skip 576 audio samples from
 curr_cts: 1024
 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x558bdd922d00] demuxer injecting skip 1600 /
 discard 0
 [aac @ 0x558bdd923e00] skip 1600 / discard 0 samples due to side data
 [aac @ 0x558bdd93a000] skip 1600 / discard 0 samples due to side data
 [aac @ 0x558bdd929e80] skip 1600 / discard 0 samples due to side data
 [aac @ 0x558bdd929e80] skip 576/1024 samples
 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x558bdd9242c0] skip 576 audio samples from
 curr_cts: 1024
 [mov,mp4,m4a,3gp,3g2,mj2 @ 0x558bdd9242c0] demuxer injecting skip 1600 /
 discard 0
 [aac @ 0x558bddb6d880] skip 1600 / discard 0 samples due to side data
 [aac @ 0x558bdd929e80] skip 1600 / discard 0 samples due to side data
 [aac @ 0x558bdd929e80] skip 576/1024 samples
 }}}

 (Opus streams on the other hand work fine - I read that they are
 inherently gapless, but YT Music provides "high-quality" streams,
 available with Premium subscriptions, in m4a only)
-- 
Ticket URL: <https://trac.ffmpeg.org/ticket/10369>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker