[FFmpeg-user] moov atom way too big + udta added

Mayescid Ocomance MayescidOcomance at gmail.com
Fri Jun 7 12:45:50 EEST 2019


Hello everyone,

I'm creating serveral mp4 derivatives of other mp4s and m2ts using ffmpeg.
It's an automated process. So far I have used this command line for one of
the derivaties:

-s 1280x720  -c:v libx264 -profile:v main -crf 10 -maxrate 2M -bufsize 2M
-c:a aac -ac 2 -b:a 128k -ar 44100 -f mp4 -movflags +faststart


The file is fine, however, when I compare its structure using AtomicParsley
to a file with the same specifics but created by another programm, I'm
getting way too big moov atoms, which seems to be caused by way too big
stscm, stszm, stco, (and ctts for audio) atoms in both streams.

(Keep in mind this is just a 2-3 minute video. The difference is even 4-5x
when encoding 1-2h videos).

Does anyone have an idea, what I could change to reduce the moov atom file ?
So far I have tried forcing keyframe intervals, bigger GOPs, disabling
metadata, maping chapters differenty, alas without any or only teeny tiny
progress in moov reduction.

(and any idea how to get rid of the udta that is not part of the original
input and seemingly is generated by ffmpeg by default ?!)


Here is a comparison:

With FFMPEG (ffmpeg output below)

 Major Brand: isom  -  version 512
 Compatible Brands: isom iso2 avc1 mp41
 Tagging schemes available:
   iTunes-style metadata allowed.
   ISO-copyright notices @ movie and/or track level allowed.
   uuid private user extension tags allowed.
Atom "©too" contains: Lavf58.23.100
---------------------------
free atom space: 8
padding available: 8 bytes
user data space: 98
media data space: 34289751
Atom ftyp @ 0 of size: 32, ends @ 32
Atom moov @ 32 of size: 113694, ends @ 113726
     Atom mvhd @ 40 of size: 108, ends @ 148
     Atom trak @ 148 of size: 54008, ends @ 54156
         Atom tkhd @ 156 of size: 92, ends @ 248
         Atom edts @ 248 of size: 36, ends @ 284
             Atom elst @ 256 of size: 28, ends @ 284
         Atom mdia @ 284 of size: 53872, ends @ 54156
             Atom mdhd @ 292 of size: 32, ends @ 324
             Atom hdlr @ 324 of size: 52, ends @ 376
             Atom minf @ 376 of size: 53780, ends @ 54156
                 Atom vmhd @ 384 of size: 20, ends @ 404
                 Atom dinf @ 404 of size: 36, ends @ 440
                     Atom dref @ 412 of size: 28, ends @ 440
                 Atom stbl @ 440 of size: 53716, ends @ 54156
                     Atom stsd @ 448 of size: 168, ends @ 616
                         Atom avc1 @ 464 of size: 152, ends @ 616
                             Atom avcC @ 550 of size: 50, ends @ 600
                             Atom pasp @ 600 of size: 16, ends @ 616 ~
                     Atom stts @ 616 of size: 24, ends @ 640
                     Atom stss @ 640 of size: 76, ends @ 716
                     Atom ctts @ 716 of size: 26504, ends @ 27220
                     Atom stsc @ 27220 of size: 40, ends @ 27260
                     Atom stsz @ 27260 of size: 13452, ends @ 40712
                     Atom stco @ 40712 of size: 13444, ends @ 54156
     Atom trak @ 54156 of size: 59472, ends @ 113628
         Atom tkhd @ 54164 of size: 92, ends @ 54256
         Atom edts @ 54256 of size: 36, ends @ 54292
             Atom elst @ 54264 of size: 28, ends @ 54292
         Atom mdia @ 54292 of size: 59336, ends @ 113628
             Atom mdhd @ 54300 of size: 32, ends @ 54332
             Atom hdlr @ 54332 of size: 52, ends @ 54384
             Atom minf @ 54384 of size: 59244, ends @ 113628
                 Atom smhd @ 54392 of size: 16, ends @ 54408
                 Atom dinf @ 54408 of size: 36, ends @ 54444
                     Atom dref @ 54416 of size: 28, ends @ 54444
                 Atom stbl @ 54444 of size: 59184, ends @ 113628
                     Atom stsd @ 54452 of size: 106, ends @ 54558
                         Atom mp4a @ 54468 of size: 90, ends @ 54558
                             Atom esds @ 54504 of size: 54, ends @ 54558
                     Atom stts @ 54558 of size: 32, ends @ 54590
                     Atom stsc @ 54590 of size: 22372, ends @ 76962
                     Atom stsz @ 76962 of size: 23168, ends @ 100130
                     Atom stco @ 100130 of size: 13444, ends @ 113574
                     Atom sgpd @ 113574 of size: 26, ends @ 113600 ~
                     Atom sbgp @ 113600 of size: 28, ends @ 113628
     Atom udta @ 113628 of size: 98, ends @ 113726
         Atom meta @ 113636 of size: 90, ends @ 113726
             Atom hdlr @ 113648 of size: 33, ends @ 113681
             Atom ilst @ 113681 of size: 45, ends @ 113726
                 Atom ©too @ 113689 of size: 37, ends @ 113726
                     Atom data @ 113697 of size: 29, ends @ 113726
Atom free @ 113726 of size: 8, ends @ 113734
Atom mdat @ 113734 of size: 34289751, ends @ 34403485

 ~ denotes an unknown atom
------------------------------------------------------
Total size: 34403485 bytes; 53 atoms total. AtomicParsley version: 0.9.0
(utf16)
Media data: 34289751 bytes; 113734 bytes all other atoms (0.331% atom
overhead).
Total free atom space: 8 bytes; 0.000% waste. Padding available: 8 bytes.

Other Programm:

 Major Brand: mp42  -  version 0
 Compatible Brands: isom mp42
 Tagging schemes available:
   iTunes-style metadata allowed.
   ISO-copyright notices @ movie and/or track level allowed.
   uuid private user extension tags allowed.
---------------------------
free atom space: 0
padding available: 0 bytes
media data space: 35721546
Atom ftyp @ 0 of size: 24, ends @ 24
Atom moov @ 24 of size: 42655, ends @ 42679
     Atom mvhd @ 32 of size: 108, ends @ 140
     Atom iods @ 140 of size: 33, ends @ 173
     Atom trak @ 173 of size: 27028, ends @ 27201
         Atom tkhd @ 181 of size: 92, ends @ 273
         Atom mdia @ 273 of size: 26928, ends @ 27201
             Atom mdhd @ 281 of size: 32, ends @ 313
             Atom hdlr @ 313 of size: 68, ends @ 381
             Atom minf @ 381 of size: 26820, ends @ 27201
                 Atom smhd @ 389 of size: 16, ends @ 405
                 Atom dinf @ 405 of size: 36, ends @ 441
                     Atom dref @ 413 of size: 28, ends @ 441
                 Atom stbl @ 441 of size: 26760, ends @ 27201
                     Atom stsd @ 449 of size: 100, ends @ 549
                         Atom mp4a @ 465 of size: 84, ends @ 549
                             Atom esds @ 501 of size: 48, ends @ 549
                     Atom stts @ 549 of size: 232, ends @ 781
                     Atom stsc @ 781 of size: 1888, ends @ 2669
                     Atom stsz @ 2669 of size: 23176, ends @ 25845
                     Atom stco @ 25845 of size: 1356, ends @ 27201
     Atom trak @ 27201 of size: 15478, ends @ 42679
         Atom tkhd @ 27209 of size: 92, ends @ 27301
         Atom mdia @ 27301 of size: 15378, ends @ 42679
             Atom mdhd @ 27309 of size: 32, ends @ 27341
             Atom hdlr @ 27341 of size: 68, ends @ 27409
             Atom minf @ 27409 of size: 15270, ends @ 42679
                 Atom vmhd @ 27417 of size: 20, ends @ 27437
                 Atom dinf @ 27437 of size: 36, ends @ 27473
                     Atom dref @ 27445 of size: 28, ends @ 27473
                 Atom stbl @ 27473 of size: 15206, ends @ 42679
                     Atom stsd @ 27481 of size: 178, ends @ 27659
                         Atom avc1 @ 27497 of size: 162, ends @ 27659
                             Atom avcC @ 27583 of size: 76, ends @ 27659
                     Atom stts @ 27659 of size: 48, ends @ 27707
                     Atom stsc @ 27707 of size: 40, ends @ 27747
                     Atom stsz @ 27747 of size: 13448, ends @ 41195
                     Atom stco @ 41195 of size: 1360, ends @ 42555
                     Atom stss @ 42555 of size: 124, ends @ 42679
Atom mdat @ 42679 of size: 35721546 (^), ends @ 0
(^)denotes a 64-bit atom length
------------------------------------------------------
Total size: 35764225 bytes; 39 atoms total. AtomicParsley version: 0.9.0
(utf16)
Media data: 35721546 bytes; 42679 bytes all other atoms (0.119% atom
overhead).
Total free atom space: 0 bytes; 0.000% waste.

----


FFMPEG Output:

C:\ffmpeg\bin\ffmpeg.exe : ffmpeg version N-92528-g90ac0e5f29 Copyright (c)
2000-2018 the FFmpeg developers
At line:1 char:1
+ C:\ffmpeg\bin\ffmpeg.exe -y -i INPUTDATA
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ffmpeg version ...mpeg
developers:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

  built with gcc 8.2.1 (GCC) 20181017
  configuration: --enable-gpl --enable-version3 --enable-sdl2
--enable-fontconfig --enable-gnutls --enable-iconv --enable-libass
--enable-libbluray --enable-libfreetype --enable-libmp3lame
--enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg
--enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr
--enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack
--enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2
--enable-libzimg --enable-lzma --enable-zlib --enable-gmp
--enable-libvidstab --enable-libvorbis --enable-libvo-amrwbenc
--enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom
--enable-libmfx --enable-amf --enable-ffnvcodec --enable-cuvid
--enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2
--enable-avisynth
  libavutil      56. 24.101 / 56. 24.101
  libavcodec     58. 40.100 / 58. 40.100
  libavformat    58. 23.100 / 58. 23.100
  libavdevice    58.  6.100 / 58.  6.100
  libavfilter     7. 46.100 /  7. 46.100
  libswscale      5.  4.100 /  5.  4.100
  libswresample   3.  4.100 /  3.  4.100
  libpostproc    55.  4.100 / 55.  4.100
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000013d7dbcae80]
 st: 0 edit list: 1 Missing key frame while searching for timestamp: 1
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000013d7dbcae80] st: 0 edit list 1 Cannot find
an index entry before timestamp: 1.
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'INPUTDATA'
    major_brand     : avc1
    minor_version   : 0
    compatible_brands: avc1mp42
    creation_time   : 2019-06-07T06:58:05.000000Z
  Duration: 00:02:14.32, start: 0.000000, bitrate: 17415 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(tv,
bt709), 1920x1080 [SAR 1:1 DAR 16:9], 17019 kb/s,
25 fps,
25 tbr, 25 tbn,
50 tbc (default)
    Metadata:
      creation_time   : 2019-06-07T06:58:05.000000Z
      handler_name    : Video Media Handler
      encoder         : AVC Coding
    Stream #0:
1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 384
kb/s (default)
    Metadata:
      creation_time   : 2019-06-07T06:58:05.000000Z
      handler_name    : Sound Media Handler
Stream mapping:
  Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
  Stream #0:1 -> #0:1 (aac (native) -> aac (native))
Press [q] to stop, [?] for help
[libx264 @ 0000013d0075a280]
 using SAR=1/1
[libx264 @ 0000013d0075a280] using cpu capabilities: MMX2 SSE2Fast SSSE3
SSE4.2
[libx264 @ 0000013d0075a280] profile Main, level 3.1, 4:2:0, 8-bit
[libx264 @ 0000013d0075a280] 264 - core 157 r2935 545de2f - H.264/MPEG-4
AVC codec - Copyleft 2003-2018 - http://www.videolan.org/x264.html -
options: cabac=1 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1
psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0
cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=6
lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0
bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1
b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25
scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=10.0
qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 vbv_maxrate=2000 vbv_bufsize=2000
crf_max=0.0 nal_hrd=none filler=0 ip_ratio=1.40 aq=1:1.00
Output #0, mp4, to 'INPUTDATA':
  Metadata:
    major_brand     : avc1
    minor_version   : 0
    compatible_brands: avc1mp42
    encoder         : Lavf58.23.100
    Stream #0:0(eng): Video: h264 (libx264) (avc1 / 0x31637661),
yuv420p(progressive), 1280x720 [SAR 1:1 DAR 16:9], q=-1--1, 25 fps, 12800
tbn, 25 tbc (default)
    Metadata:
      creation_time   : 2019-06-07T06:58:05.000000Z
      handler_name    : Video Media Handler
      encoder         : Lavc58.40.100 libx264
    Side data:
      cpb: bitrate max/min/avg: 2000000/0/0 buffer size: 2000000 vbv_delay:
-1
    Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz,
stereo, fltp, 128 kb/s (default)
    Metadata:
      creation_time   : 2019-06-07T06:58:05.000000Z
      handler_name    : Sound Media Handler
      encoder         : Lavc58.40.100 aac
frame=   22 fps=0.0 q=0.0 size=       0kB time=00:00:00.90 bitrate=
0.4kbits/s speed=1.76x
frame=   47 fps= 33 q=0.0 size=       0kB time=00:00:01.85 bitrate=
0.2kbits/s speed=1.32x
frame=   58 fps= 30 q=25.0 size=       0kB time=00:00:02.34 bitrate=
0.2kbits/s speed=1.22x
frame=   70 fps= 28 q=25.0 size=       0kB time=00:00:02.83 bitrate=
0.1kbits/s speed=1.15x
frame=   79 fps= 26 q=25.0 size=     256kB time=00:00:03.08 bitrate=
679.2kbits/s speed=1.03x
...
frame= 3292 fps= 30 q=25.0 size=   32256kB time=00:02:11.72
bitrate=2006.0kbits/s speed=1.19x
frame= 3303 fps= 30 q=26.0 size=   32256kB time=00:02:12.19
bitrate=1998.9kbits/s speed=1.19x
frame= 3315 fps= 30 q=26.0 size=   32512kB time=00:02:12.67
bitrate=2007.4kbits/s speed=1.19x
frame= 3327 fps= 30 q=27.0 size=   32512kB time=00:02:13.14
bitrate=2000.4kbits/s speed=1.19x
frame= 3337 fps= 30 q=27.0 size=   32768kB time=00:02:13.39
bitrate=2012.3kbits/s speed=1.19x
frame= 3352 fps= 30 q=27.0 size=   32768kB time=00:02:14.14
bitrate=2001.1kbits/s speed=1.19x
[mp4 @ 0000013d7fafe840]
 Starting second pass: moving the moov atom to the beginning of the file
frame= 3358 fps= 29 q=-1.0 Lsize=   33653kB time=00:02:14.35
bitrate=2052.0kbits/s speed=1.17x
video:31405kB audio:2138kB subtitle:0kB other streams:0kB global
headers:0kB muxing overhead: 0.331155%
[libx264 @ 0000013d0075a280]
 frame I:15    Avg QP:17.47  size:127527
[libx264 @ 0000013d0075a280] frame P:899   Avg QP:18.98  size: 26307
[libx264 @ 0000013d0075a280] frame B:2444  Avg QP:22.55  size:  2698
[libx264 @ 0000013d0075a280] consecutive B-frames:  0.6%  5.8%  3.8% 89.8%
[libx264 @ 0000013d0075a280] mb I  I16..4: 18.6%  0.0% 81.4%
[libx264 @ 0000013d0075a280] mb P  I16..4:  0.2%  0.0%  1.2%  P16..4: 48.5%
21.3% 16.6%  0.0%  0.0%    skip:12.2%
[libx264 @ 0000013d0075a280] mb B  I16..4:  0.0%  0.0%  0.1%  B16..8: 36.4%
 2.1%  0.5%  direct: 0.8%  skip:60.2%  L0:38.3% L1:56.5% BI: 5.2%
[libx264 @ 0000013d0075a280] coded y,uvDC,uvAC intra: 90.9% 89.8% 58.4%
inter: 12.4% 21.1% 1.1%
[libx264 @ 0000013d0075a280] i16 v,h,dc,p: 11% 20% 21% 48%
[libx264 @ 0000013d0075a280] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 16% 26% 12%  6%
 7%  7% 10%  6% 11%
[libx264 @ 0000013d0075a280] i8c dc,h,v,p: 42% 29% 18% 11%
[libx264 @ 0000013d0075a280] Weighted P-Frames: Y:0.2% UV:0.0%
[libx264 @ 0000013d0075a280] ref P L0: 61.1% 11.8% 19.5%  7.6%  0.0%
[libx264 @ 0000013d0075a280] ref B L0: 92.7%  6.2%  1.2%
[libx264 @ 0000013d0075a280] ref B L1: 95.3%  4.7%
[libx264 @ 0000013d0075a280] kb/s:1915.28
[aac @ 0000013d004200c0]
 Qavg: 351.123



Thanks, Mayescid


More information about the ffmpeg-user mailing list