first keyframe is missed in h264 mpegts stream

[DonMoir]

Every mpegts file I have has some problem. Most of these problems are minor. Generally, I only notice the problems when seeking and have CODEC_FLAG2_SHOW_ALL set. Mostly this will cause a slight distortion in the display. If CODEC_FLAG2_SHOW_ALL is not set, then a slight delay will occur but not really noticeable.

I think that all playback of any mpegts file will cause some error to be displayed by ffplay even though actual playback is ok. I suppose it has recovered reasonably well in some cases.

On some occasions though, the playback and seek is bad for mpegts files.

This file is one of the bad cases:

 http://sms.pangolin.com/temp/mpegts_problem_file.ts

In SMPLayer and ffplay, it will skip some initial frames on startup. Seeking does not work at all with SMPlayer for this file. With ffplay seeking does work, but if you keep seeking on it, audio will be out of sync. On startup you should see the just the lady standing there. With ffplay and SMPlayer it skips the first part and goes right to the lady standing and the words REQUIEM etc, being displayed.

For my own app and with CODEC_FLAG2_SHOW_ALL set, I will see gray frames on startup and when seeking. When CODEC_FLAG2_SHOW_ALL is not set, I see delays.

The VLC player shows distortion on startup and when seeking.

The thing is, ffdshow in WMP, gets this file absolutely correct both on startup and when seeking. That is, there is no distortion, gray frames, or delays.

This is the worse case file I have, but I have others with problems. All of them really, but most cases are minor, but some more severe.

The errors reported by ffplay, mostly come from h264.c and h264_parser.c.

I think this file just may be a worse indication of a general problem with the mpegts demuxer and other related code. The fact that ffdshow has no problem at all with the above file leads me to believe this.

Keep in mind, all ts files I tested have some sort of a problem. Most of these are minor and only when seeking. Like it did not seek to an actual keyframe, but recovery is fast for the most part.

The seeking issues may be related to the above or just bad keyframe management in libavformat/mpegts.c. The read_timestamp function for mpegts (mpegts_get_dts), has a FIXME next to it's av_add_index_entry.

Here's the ffplay playback startup output:

ffplay mpegts_problem_file.ts

ffplay version N-38292-ga4c22e3 Copyright (c) 2003-2012 the FFmpeg developers
  built on Feb 27 2012 14:52:55 with gcc 4.6.2
  configuration: --disable-static --enable-shared --enable-gpl --enable-version3
 --disable-w32threads --enable-runtime-cpudetect --enable-avisynth
 --enable-bzlib --enable-frei0r --enable-libopencore-amrnb --enable-libopencore-amrwb
 --enable-libfreetype --enable-libgsm --enable-libmp3lame --enable-libopenjpeg
 --enable-librtmp --enable-libschroedinger --enable-libspeex --enable-libtheora
 --enable-libvo-aacenc --enable-libvo-amrwbenc --enable-libvorbis --enable-libvpx
 --enable-libx264 --enable-libxavs --enable-libxvid --enable-zlib
  libavutil      51. 41.100 / 51. 41.100
  libavcodec     54.  4.100 / 54.  4.100
  libavformat    54.  1.100 / 54.  1.100
  libavdevice    53.  4.100 / 53.  4.100
  libavfilter     2. 62.101 /  2. 62.101
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0.  7.100 /  0.  7.100
  libpostproc    52.  0.100 / 52.  0.100
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] non-existing PPS referenced
[h264 @ 014C17C0] non-existing PPS 0 referenced
[h264 @ 014C17C0] decode_slice_header error
[h264 @ 014C17C0] no frame!
[h264 @ 014C17C0] Increasing reorder buffer to 1
[h264 @ 014C17C0] Increasing reorder buffer to 2
[mpegts @ 00A4D8A0] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 00A4D8A0] PES packet size mismatch
Input #0, mpegts, from 'mpegts_problem_file.ts':
  Duration: 00:00:25.36, start: 7140.559933, bitrate: 1875 kb/s
  Program 5041
  Program 5042
  Program 5043
  Program 5044
  Program 5045
  Program 5047
  Program 5048
  Program 5049
  Program 5040
  Program 5056
  Program 5046
  Program 5061
  Program 5064
  Program 5065
  Program 5058
  Program 5059
    Stream #0:0[0xfb3]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 70
4x576 [SAR 12:11 DAR 4:3], 28.61 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x1053](cze): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, ster
eo, s16, 192 kb/s
  Program 5057
[h264 @ 017D9040] reference picture missing during reorder
    Last message repeated 1 times
[h264 @ 017D9040] Missing reference picture
[h264 @ 017D9040] decode_slice_header error
[h264 @ 017D9040] concealing 1584 DC, 1584 AC, 1584 MV errors
[h264 @ 017D9040] reference picture missing during reorder=    0B f=0/0
[h264 @ 017D9040] Missing reference picture
7143.33 A-V: -2.196 fd=   0 aq=  319KB vq= 1901KB sq=    0B f=0/0   f=0/0

[DonMoir]

The startup_display.png image just shows what image is supposed to be displayed when playback begins. With ffplay, it has skipped about a second. There is more wrong then just this though. Again, ffdshow gets the above mpegts_problem_file perfect in all respects.

This link will give you a quicker look at startup_display.png.

 http://sms.pangolin.com/temp/startup_display.png

[cehoyos]

The following ffmpeg command line produces a keyframe, compared with the following mplayer command line, it can be seen that it is the second keyframe of the stream.

$ ffmpeg -i mpegts_problem_file.ts -vframes 1 out.png
ffmpeg version N-42870-g1c2beff Copyright (c) 2000-2012 the FFmpeg developers
  built on Jul 25 2012 09:12:59 with gcc 4.3.2 (GCC)
  configuration: --cc=/usr/local/gcc-4.3.2/bin/gcc --enable-gpl --enable-libopenjpeg --enable-libvorbis --enable-libspeex --enable-libmp3lame --enable-libtheora --extra-ldflags=-lm --enable-libvpx --enable-libxavs --enable-x11grab --enable-libass --enable-nonfree --enable-libfaac --enable-libopus
  libavutil      51. 65.100 / 51. 65.100
  libavcodec     54. 44.100 / 54. 44.100
  libavformat    54. 20.100 / 54. 20.100
  libavdevice    54.  2.100 / 54.  2.100
  libavfilter     3.  3.100 /  3.  3.100
  libswscale      2.  1.100 /  2.  1.100
  libswresample   0. 15.100 /  0. 15.100
  libpostproc    52.  0.100 / 52.  0.100
[h264 @ 0x90cebc0] non-existing PPS referenced
[h264 @ 0x90cebc0] non-existing PPS 0 referenced
[h264 @ 0x90cebc0] decode_slice_header error
[h264 @ 0x90cebc0] no frame!

[...]

[h264 @ 0x90cebc0] non-existing PPS referenced
[h264 @ 0x90cebc0] non-existing PPS 0 referenced
[h264 @ 0x90cebc0] decode_slice_header error
[h264 @ 0x90cebc0] no frame!
[mpegts @ 0x90b6500] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x90b6500] PES packet size mismatch
Input #0, mpegts, from 'mpegts_problem_file.ts':
  Duration: 00:00:25.36, start: 7140.559933, bitrate: 1875 kb/s
  Program 5041
  Program 5042
  Program 5043
  Program 5044
  Program 5045
  Program 5047
  Program 5048
  Program 5049
  Program 5040
  Program 5056
  Program 5046
  Program 5061
  Program 5064
  Program 5065
  Program 5058
  Program 5059
    Stream #0:0[0xfb3]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 704x576 [SAR 12:11 DAR 4:3], 28.61 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x1053](cze): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16, 192 kb/s
  Program 5057
Output #0, image2, to 'out.png':
  Metadata:
    encoder         : Lavf54.20.100
    Stream #0:0: Video: png, rgb24, 704x576 [SAR 12:11 DAR 4:3], q=2-31, 200 kb/s, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (h264 -> png)
Press [q] to stop, [?] for help
[h264 @ 0x90cebc0] reference picture missing during reorder
    Last message repeated 1 times
[h264 @ 0x90cebc0] Missing reference picture
[h264 @ 0x90cebc0] decode_slice_header error
[h264 @ 0x90cebc0] concealing 1584 DC, 1584 AC, 1584 MV errors in P frame
[h264 @ 0x90cebc0] reference picture missing during reorder
[h264 @ 0x90cebc0] Missing reference picture
frame=    1 fps=0.0 q=0.0 Lsize=       0kB time=00:00:00.04 bitrate=   0.0kbits/s dup=0 drop=2
video:435kB audio:0kB subtitle:0 global headers:0kB muxing overhead -100.000000%

The following command produces 19 frames, frame 00000007.png (- 000000018.png) shows the first keyframe, 00000019.png the second one (that ffmpeg shows above).

$ mplayer mpegts_problem_file.ts -demuxer lavf -lavdopts skipframe=nonkey -vo png -frames 19

[DonMoir]

Thanks for getting this opened Carl. The mpegts_problem_file.ts file represents the most in the way of playback / seek problems I see currently. Still, it is just a few mpegts-h264 files that are this bad, but there are indications that all mpegts-h264 have a problem when seeking. In most cases this is very minor and barely noticeable.

If it can be found why the first frames are skipped, it may lead to reasons why seeking can also be bad for mpegts-h264 files. It is likely that seek problems are related to the way key entries are being managed.

Again, apparently the mpegts_problem_file.ts file is not a bad file, but various players have problems with it. ffdshow in WMP got it perfect.