MP4 remux with playback stuck at 2nd seconds

[mikhailov]

Summary of the bug:

We use FLV captured stream from webcam (using application build for Flash Player), h264 comes direct from the flash, but audio stream Speex should be encoded to AAC to make the streams compatible with mobile devices h264 (Baseline) + AAC.

Video stream should not be encoded again, so we use the copy method, like the following:

ffmpeg -i input.flv -c:v copy -c:a libvo_aacenc output.mp4

The resulted MP4 video files plays back well on Flash Player 11.4+ and iPhone/iPad with iOS 5+ (Baseline compatible video stream). but on Android devices and Flash Player 11.1/11.2 the playback freezes at 2-3 seconds. We also use qt-faststart, but it doesn't affect on stuck problem.

Stdout of ffmpeg encoding script:

ffmpeg version 1.0 Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov 17 2012 23:45:34 with Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/1.0 --enable-shared --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --cc=cc --host-cflags= --host-ldflags= --enable-libx264 --enable-libfaac --enable-libvo-aacenc --enable-libspeex
  libavutil      51. 73.101 / 51. 73.101
  libavcodec     54. 59.100 / 54. 59.100
  libavformat    54. 29.104 / 54. 29.104
  libavdevice    54.  2.101 / 54.  2.101
  libavfilter     3. 17.100 /  3. 17.100
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[libspeex @ 0x7fabb902ea00] Missing Speex header, assuming defaults.
Guessed Channel Layout for  Input Stream #0.1 : mono
Input #0, flv, from '1.flv':
  Metadata:
    creationdate    : Thu Nov 22 09:46:36
    duration2       : 29964
  Duration: 00:00:30.25, start: 0.000000, bitrate: 212 kb/s
    Stream #0:0: Video: h264 (Baseline), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 20 tbr, 1k tbn, 40 tbc
    Stream #0:1: Audio: speex, 16000 Hz, mono, s16
File '1.mp4' already exists. Overwrite ? [y/N] y
Missing Speex header, assuming defaults.
Output #0, mp4, to '1.mp4':
  Metadata:
    creationdate    : Thu Nov 22 09:46:36
    duration2       : 29964
    encoder         : Lavf54.29.104
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 1k tbn, 1k tbc
    Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 16000 Hz, mono, s16, 96 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (libspeex -> libvo_aacenc)
Press [q] to stop, [?] for help
[mp4 @ 0x7fabb9012800] st:0 PTS: 2139 DTS: 2139 < 2140 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 2722 DTS: 2722 < 2723 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 5440 DTS: 5440 < 5441 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 8105 DTS: 8105 < 8106 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 10772 DTS: 10772 < 10773 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 13456 DTS: 13456 < 13457 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 16127 DTS: 16127 < 16128 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 18835 DTS: 18835 < 18836 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 21675 DTS: 21675 < 21676 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 24476 DTS: 24476 < 24477 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 27283 DTS: 27283 < 27284 invalid, clipping
[mp4 @ 0x7fabb9012800] st:0 PTS: 30079 DTS: 30079 < 30080 invalid, clipping
frame=  454 fps=0.0 q=-1.0 Lsize=     978kB time=00:00:30.31 bitrate= 264.4kbits/s    
video:620kB audio:346kB subtitle:0 global headers:0kB muxing overhead 1.170369%

I tried different versions of ffmpeg (0.10, 0.11, 1.0, 1.01) compiled with mandatory --with-libvo-aacenc --with-speex, but the problem still not resolved.

[cehoyos]

Are you sure you are reporting this on the right bug tracker?

Please test current git head.

Is this only reproducible with -acodec libvo_aacenc or also with -acodec libfaac and -strict -2 -acodec aac? What about -an?

[mikhailov]

I would be happy if you can advice me another right ffmpeg bug tracker.

Yes, I tried multiple audio codecs and even delete audio stream while remuxing (-an), the result is the same.
The problem with demux process from FLV to MP4 container (Video stream still untouchable).

[cehoyos]

Replying to mikhailov:

I would be happy if you can advice me another right ffmpeg bug tracker.

I meant if the video plays on an iPhone but not on Android devices, this may indicate that the problem can only be fixed on the Android devices.

Please test current git head and -an (and post console output). If the resulting video is not played, I wonder if this wouldn't indicate that the bitrate (level) of the stream is too high.

[mikhailov]

test file with FLV container (h264 + Speex)  http://yadi.sk/d/75d7DWh01YPxf

[cehoyos]

Please run "ffmpeg -i 302_1991_7668.flv -vcodec copy -an out.mp4" using current git head and test the resulting file on your Android device.

[mikhailov]

yes, I'll fetch git HEAD, compile ffmpeg and perform the encoding with it very soon

[mikhailov]

I've got the git HEAD:

git clone --depth=1 git://source.ffmpeg.org/ffmpeg.git
./configure --enable-gpl --enable-libx264 --enable-libmp3lame --enable-nonfree --enable-libvo-aacenc --enable-libspeex --enable-version3
make

then tried to remux the container:

ffmpeg -i 1.flv -vcodec copy -an out.mp4

the result the same: playback fine on OS X and iPhone/iPad, but stuck on 2sec. when you try to playback on Flash 11.1 (flowplayer) or Android devices:

 http://yadi.sk/d/YQux1bCq1YVJV

How can I check the MP4 container, is it valid or not?

ffmpeg -i 1.flv -vcodec copy -an out.mp4
ffmpeg version git-2012-12-31-9827528 Copyright (c) 2000-2012 the FFmpeg developers
  built on Dec 31 2012 13:16:36 with llvm-gcc 4.2.1 (LLVM build 2336.11.00)
  configuration: --enable-gpl --enable-libx264 --enable-libmp3lame --enable-nonfree --enable-libvo-aacenc --enable-libspeex --enable-version3
  libavutil      52. 13.100 / 52. 13.100
  libavcodec     54. 85.100 / 54. 85.100
  libavformat    54. 58.100 / 54. 58.100
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 30.102 /  3. 30.102
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
Input #0, flv, from '1.flv':
  Metadata:
    creationdate    : Thu Nov 22 09:46:36
    duration2       : 29964
  Duration: 00:00:30.25, start: 0.000000, bitrate: 212 kb/s
    Stream #0:0: Video: h264 (Baseline), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 20 tbr, 1k tbn, 40 tbc
    Stream #0:1: Audio: speex, 16000 Hz, mono, s16
Output #0, mp4, to 'out.mp4':
  Metadata:
    creationdate    : Thu Nov 22 09:46:36
    duration2       : 29964
    encoder         : Lavf54.58.100
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=2-31, 16k tbn, 1k tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Press [q] to stop, [?] for help
[mp4 @ 0x7f9d13837600] st:0 PTS: 34224 DTS: 34224 < 34225 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 43552 DTS: 43552 < 43553 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 87040 DTS: 87040 < 87041 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 129680 DTS: 129680 < 129681 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 172352 DTS: 172352 < 172353 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 215296 DTS: 215296 < 215297 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 258032 DTS: 258032 < 258033 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 301360 DTS: 301360 < 301361 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 346800 DTS: 346800 < 346801 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 391616 DTS: 391616 < 391617 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 436528 DTS: 436528 < 436529 invalid, clipping
[mp4 @ 0x7f9d13837600] st:0 PTS: 481264 DTS: 481264 < 481265 invalid, clipping
frame=  454 fps=0.0 q=-1.0 Lsize=     626kB time=00:00:30.25 bitrate= 169.6kbits/s    
video:620kB audio:0kB subtitle:0 global headers:0kB muxing overhead 0.985025%

[mikhailov]

Each video file we put into the queue through ffmpeg and qt-faststart, everything with according to documentation. But the MP4 container becomes not full compatible, I don't know the reason why Flash Player < 11.1 and Android devices can't play back it well (normally h264 works with Flash Player 9.0+).

When we use libx264 to transcode the video stream (h264+AAC) plays back fine on every Flash Player and Android devices(!), but when remux problem is starting to happen.

[mikhailov]

What would you advice me to try? I can provide more file samples if necessary.

[cehoyos]

Replying to mikhailov:

When we use libx264 to transcode the video stream (h264+AAC) plays back fine on every Flash Player and Android devices(!), but when remux problem is starting to happen.

Could you provide a x264-encoded sample that plays fine with Flash and Android, and show the ffmpeg remuxing command line (together with complete, uncut console output) that produces a failing sample?

[mikhailov]

x264 sample to playback fine on Androids is able to get through encdoding with libx264, but not remuxing. I can apply libx264 result file, but it's a normal valid h264+AAC file.

[mikhailov]

Just take the test file with FLV container (h264 + Speex)  http://yadi.sk/d/75d7DWh01YPxf
and run

ffmpeg -i test.flv -c:v libx264 -profile:v baseline -c:a libvo_aacenc -ab 96k -ac 1 result.mp4

It will produce correct MP4+AAC file can be played back on any devices.

[cehoyos]

How can this sample be used to produce a file that does not work on your device?

[mikhailov]

Replying to cehoyos:

How can this sample be used to produce a file that does not work on your device?

via remux:

ffmpeg -i 1.flv -vcodec copy -an out.mp4

[cehoyos]

If result.mp4 plays fine, does the following command produce an output file that can be played with your device?
$ ffmpeg -i result.mp4 -acodec copy -vcodec copy out.mp4

[mikhailov]

Replying to cehoyos:

If result.mp4 plays fine, does the following command produce an output file that can be played with your device?
$ ffmpeg -i result.mp4 -acodec copy -vcodec copy out.mp4

I wrote about it before, copy (container remux) broke the file. MP4Box manually demux-mux works fine, but MP4Box is very complex and out-of-date software and it's very hard to compile on CentOS.

[cehoyos]

(Assuming that non-native speakers may read this ticket, one could argue that you failed to answer the question.)

Please provide result.mp4.

[mikhailov]

here you go:

 http://yadi.sk/d/70msco_v1t3zk

ffmpeg -i 1.flv -c:v libx264 -profile:v baseline -c:a libvo_aacenc -ab 96k -ac 1 result.mp4
ffmpeg version 1.0 Copyright (c) 2000-2012 the FFmpeg developers
  built on Nov 17 2012 23:45:34 with Apple clang version 4.1 (tags/Apple/clang-421.11.66) (based on LLVM 3.1svn)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/1.0 --enable-shared --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --cc=cc --host-cflags= --host-ldflags= --enable-libx264 --enable-libfaac --enable-libvo-aacenc --enable-libspeex
  libavutil      51. 73.101 / 51. 73.101
  libavcodec     54. 59.100 / 54. 59.100
  libavformat    54. 29.104 / 54. 29.104
  libavdevice    54.  2.101 / 54.  2.101
  libavfilter     3. 17.100 /  3. 17.100
  libswscale      2.  1.101 /  2.  1.101
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[libspeex @ 0x7f8bf982ea00] Missing Speex header, assuming defaults.
Guessed Channel Layout for  Input Stream #0.1 : mono
Input #0, flv, from '1.flv':
  Metadata:
    creationdate    : Thu Nov 22 09:46:36
    duration2       : 29964
  Duration: 00:00:30.25, start: 0.000000, bitrate: 212 kb/s
    Stream #0:0: Video: h264 (Baseline), yuv420p, 640x480 [SAR 1:1 DAR 4:3], 20 tbr, 1k tbn, 40 tbc
    Stream #0:1: Audio: speex, 16000 Hz, mono, s16
[libx264 @ 0x7f8bf9818a00] using SAR=1/1
[libx264 @ 0x7f8bf9818a00] using cpu capabilities: MMX2 SSE2Fast SSSE3 FastShuffle SSE4.1 Cache64
[libx264 @ 0x7f8bf9818a00] profile Constrained Baseline, level 3.0
[libx264 @ 0x7f8bf9818a00] 264 - core 125 - H.264/MPEG-4 AVC codec - Copyleft 2003-2012 - http://www.videolan.org/x264.html - options: cabac=0 ref=3 deblock=1:0:0 analyse=0x1:0x111 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=0 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=0 weightp=0 keyint=250 keyint_min=20 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00
[libspeex @ 0x7f8bf982ea00] Missing Speex header, assuming defaults.
Output #0, mp4, to 'result.mp4':
  Metadata:
    creationdate    : Thu Nov 22 09:46:36
    duration2       : 29964
    encoder         : Lavf54.29.104
    Stream #0:0: Video: h264 ([33][0][0][0] / 0x0021), yuv420p, 640x480 [SAR 1:1 DAR 4:3], q=-1--1, 20 tbn, 20 tbc
    Stream #0:1: Audio: aac ([64][0][0][0] / 0x0040), 16000 Hz, mono, s16, 96 kb/s
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> libx264)
  Stream #0:1 -> #0:1 (libspeex -> libvo_aacenc)
Press [q] to stop, [?] for help
frame=  606 fps= 66 q=-1.0 Lsize=    1147kB time=00:00:30.31 bitrate= 310.0kbits/s dup=163 drop=0    
video:790kB audio:346kB subtitle:0 global headers:0kB muxing overhead 0.906209%
[libx264 @ 0x7f8bf9818a00] frame I:3     Avg QP:17.11  size: 14229
[libx264 @ 0x7f8bf9818a00] frame P:603   Avg QP:21.38  size:  1270
[libx264 @ 0x7f8bf9818a00] mb I  I16..4: 47.6%  0.0% 52.4%
[libx264 @ 0x7f8bf9818a00] mb P  I16..4:  1.9%  0.0%  0.9%  P16..4: 16.0%  2.8%  0.6%  0.0%  0.0%    skip:77.7%
[libx264 @ 0x7f8bf9818a00] coded y,uvDC,uvAC intra: 32.3% 48.5% 12.4% inter: 2.8% 6.4% 0.3%
[libx264 @ 0x7f8bf9818a00] i16 v,h,dc,p: 36% 15%  8% 42%
[libx264 @ 0x7f8bf9818a00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 33% 18% 20%  5%  5%  6%  5%  5%  3%
[libx264 @ 0x7f8bf9818a00] i8c dc,h,v,p: 46% 20% 23% 11%
[libx264 @ 0x7f8bf9818a00] ref P L0: 88.8%  7.5%  3.7%
[libx264 @ 0x7f8bf9818a00] kb/s:213.44

[mikhailov]

Could you explain a bit more about 'invalid, clipping' please? Might it be the reason?

The MP4 file (copied video stream) container plays back fine during first 2-3 seconds, afteward it freezes, but sound still goes on!

[mikhailov]

FFMPEG 0.8-0.9 might work fine with container remux, just remember that found it in mailing-list, but can't find it again... Anything been changed over the last few minor releases?

[mikhailov]

Maybe I should try MP4Box or alternatives to solve the container remux issue...

[mikhailov]

any plan to fix the Mux-Demux in FFMPEG?

[cehoyos]

Replying to mikhailov:

FFMPEG 0.8-0.9 might work fine with container remux

Sorry, I have missed this message:
Could you test FFmpeg 0.8 and 0.9? If it works with one of them, it will be relatively easy to fix your problem.

[mikhailov]

Replying to cehoyos:

Replying to mikhailov:

FFMPEG 0.8-0.9 might work fine with container remux

Sorry, I have missed this message:
Could you test FFmpeg 0.8 and 0.9? If it works with one of them, it will be relatively easy to fix your problem.

It seems the Wowza Media Server bug, wrong meta-data in the files within MP4 container, reference to:
 http://www.wowza.com/forums/showthread.php?20328-Wowza-2-can-t-save-file-to-mp4!/page2

But there is a bug on FFMPEG side with FLV container, with F4V everything is ok.