[FFmpeg-user] SAR/DAR changes are not preserved with H264 in MP4 container

[Marcus Wichelmann]

Hi,

I'm using ffmpeg to encode a H264 HLS stream with fMP4 segments. Because 
my input is a DVB stream, the pixel aspect ratio of the video changes 
sometimes, for example when an old 4:3 movie is interrupted by a 
commercial. But these changes in the SAR/DAR values are not included in 
the resulting fMP4 files which causes the playback to be squeezed or 
stretched sometimes.

To reproduce this in a more simple setup, I recorded a short test clip 
which contains two aspect ratio changes:
- SAR: 16/15, DAR: 4/3 ==> SAR: 64/45, DAR: 16/9
- SAR: 64/45, DAR: 16/9 ==> SAR: 16/15, DAR: 4/3
The resolution is 720x576. You can download it here: 
https://drive.wichelmann.cloud/s/eE2oc9XikZDBzy2 
<https://drive.wichelmann.cloud/s/eE2oc9XikZDBzy2>

When playing this .ts file with ffplay or VLC, the aspect ratio change 
works fine.

When re-encoding the file with libx264 or nvenc_h264 to a new .ts file, 
everything is still fine and the aspect ratio change works when playing 
transcoded.ts:

    ffmpeg -loglevel debug -i test.ts -c:v libx264 -b:v 3M transcoded.ts


Also, the libx264 and nvenc encoder logs show that the aspect ratio 
change was detected and handled:

    ...
    [mpeg2video @ 0x3f8bfc0] Format yuv420p chosen by get_format().
    [nvenc_h264 @ 0x4013ec0] aspect ratio change (DAR): 4:3 -> 16:9
    [mpeg2video @ 0x3f8bfc0] Format yuv420p chosen by get_format().
    [nvenc_h264 @ 0x4013ec0] aspect ratio change (DAR): 16:9 -> 4:3
    ...

So obviously, the encoders are not the issue here, because they have 
support for these aspect ratio changes: 
https://git.ffmpeg.org/gitweb/ffmpeg.git/blob/HEAD:/libavcodec/nvenc.c#l2189

But when re-encoding the file to the MP4 format instead, things look 
different:

    ffmpeg -loglevel debug -i test.ts -c:v libx264 -b:v 3M transcoded.mp4

Now, when playing this transcoded.mp4 in ffplay or VLC, the 16:9 
commercial is squeezed to 4:3 when the aspect ratio changes mid-stream, 
because the SAR/DAR information is not preserved.
Same happens with vcodec copy.

Do you have an idea what might be causing this? Is this not supported 
with MP4 containers or is it just a bug in ffmpeg? (My ffmpeg is 
compiled from latest master.)

Thank you!