EIA-608 / EIA-708 Closed Captions disappear when transcoding/reencoding

[mikecheat04]

Summary of the bug:
When transoding/reencoding video ffmepg loses the CC data that was embedded within the actual video stream itself. This type of CC is referred to as EIA-608/EIA-708 and is muxed to the video stream following guidelines in SCTE 128 from my research. If you use '-c:v copy' the CC remains intact. I'm capturing live video from the gige port of a Motorola DSR-6100 IRD that is putting out UDP multicast TS.

This PDF give more details regarding how this method of CC works:
 http://www.evertz.com/resources/eia_608_708_cc.pdf

Here are the files produced from the below command. They are larger than the 10MB requested so I've hosted them on my site, here are the direct links.

 http://mikecheat.com/disjrhd.ts - Original(18.2MB)
 http://mikecheat.com/disjrsd.ts - Reencoded(8.9MB)

There is another open source project that seems to have figured out how to pull this CC data from video. They have the source code on their website, I've included the link.
 http://zapping.sourceforge.net/ZVBI/index.html

How to reproduce:
root@hdmux:/home/mike# ffmpeg -i 'udp://239.1.1.3:6100?fifo_size=9000000' -map 0:p:1:0 -c:v mpeg2video -s 704x480 -r ntsc -b:v 3000k -map 0:p:1:1 -c:a mp3 -ac 2 -ar 48000 -b:a 128k -f mpegts disjrsd.ts -map 0:p:1:0 -c:v copy -map 0:p:1:1 -c:a copy -f mpegts disjrhd.ts
ffmpeg version 1.0 Copyright (c) 2000-2012 the FFmpeg developers

built on Sep 28 2012 14:24:44 with gcc 4.4.5 (Debian 4.4.5-8)
configuration: --enable-gpl --enable-nonfree --enable-shared --enable-runtime-cpudetect --enable-libmp3lame --enable-libx264
libavutil 51. 73.101 / 51. 73.101
libavcodec 54. 59.100 / 54. 59.100
libavformat 54. 29.104 / 54. 29.104
libavdevice 54. 2.101 / 54. 2.101
libavfilter 3. 17.100 / 3. 17.100
libswscale 2. 1.101 / 2. 1.101
libswresample 0. 15.100 / 0. 15.100
libpostproc 52. 0.100 / 52. 0.100

[mpegts @ 0x2169240] Unable to seek back to the start
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] non-existing PPS referenced
[h264 @ 0x2193a80] non-existing PPS 0 referenced
[h264 @ 0x2193a80] decode_slice_header error
[h264 @ 0x2193a80] no frame!
[h264 @ 0x2193a80] mmco: unref short failure

Last message repeated 2 times

[mpegts @ 0x2169240] max_analyze_duration 5000000 reached at 5003333
[mpegts @ 0x2169240] Estimating duration from bitrate, this may be inaccurate
Input #0, mpegts, from 'udp://239.1.1.3:6100?fifo_size=9000000':

Duration: N/A, start: 11940.555644, bitrate: 768 kb/s
Program 1

Stream #0:0[0x1e00]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 61.76 fps, 59.94 tbr, 90k tbn, 119.88 tbc
Stream #0:1[0x1020](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), s16, 384 kb/s
Stream #0:2[0x1021](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, s16, 192 kb/s
Stream #0:3[0x1022](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, s16, 192 kb/s

File 'disjrhd.ts' already exists. Overwrite ? [y/N] y
muxrate VBR, pcr every 2 pkts, sdt every 200, pat/pmt every 40 pkts
[mpegts @ 0x22aad60] muxrate VBR, pcr every 5 pkts, sdt every 200, pat/pmt every 40 pkts
Output #0, mpegts, to 'disjrsd.ts':

Metadata:

encoder : Lavf54.29.104
Stream #0:0: Video: mpeg2video, yuv420p, 704x480 [SAR 40:33 DAR 16:9], q=2-31, 3000 kb/s, 90k tbn, 29.97 tbc
Stream #0:1(eng): Audio: mp3, 48000 Hz, stereo, s16, 128 kb/s

Output #1, mpegts, to 'disjrhd.ts':

Metadata:

encoder : Lavf54.29.104
Stream #1:0: Video: h264 ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], q=2-31, 61.76 fps, 90k tbn, 59.94 tbc
Stream #1:1(eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), 384 kb/s

Stream mapping:

Stream #0:0 -> #0:0 (h264 -> mpeg2video)
Stream #0:1 -> #0:1 (ac3 -> libmp3lame)
Stream #0:0 -> #1:0 (copy)
Stream #0:1 -> #1:1 (copy)

Press [q] to stop, ? for help
[h264 @ 0x28a5740] Missing reference picture
[h264 @ 0x28a5740] decode_slice_header error
[h264 @ 0x28a5740] concealing 3600 DC, 3600 AC, 3600 MV errors in B frame
[h264 @ 0x28a63a0] Missing reference picture
[h264 @ 0x28a63a0] decode_slice_header error
[h264 @ 0x28a63a0] concealing 3600 DC, 3600 AC, 3600 MV errors in B frame
[h264 @ 0x28a68e0] reference picture missing during reorder
[h264 @ 0x28a68e0] Missing reference picture
[h264 @ 0x28a68e0] decode_slice_header error
[h264 @ 0x28a68e0] concealing 3600 DC, 3600 AC, 3600 MV errors in P frame
[h264 @ 0x28a7360] mmco: unref short failure
[h264 @ 0x2c4ddc0] mmco: unref short failure
frame= 656 fps= 47 q=2.0 Lq=-1.0 size= 9102kB time=00:00:21.93 bitrate=3398.6kbits/s dup=22 drop=628
video:24157kB audio:1326kB subtitle:0 global headers:0kB muxing overhead -64.284273%
Received signal 2: terminating.

You can't use my command exactly as is because I'm capturing from a live source. You can use one of the above files to emulate the problem. If any other info is needed please let me know.

Thank you

[mikecheat04]

PS I can provide as many samples as needed. This is only one feed of about 30 that I have access to.

[cehoyos]

FFmpeg does not support closed captions, see also tickett #846.

[mikecheat04]

Wow all this time I was thinking it did. Made the assumption that subtitles and closed caption were the same thing. Okay well thanks for the update.

[reimar]

The situation is not quite as simple.
The problem in this specific case is that it does not have CC data in a separate stream but embedded in the video frame data.
FFmpeg currently has no way of handling this mess where one stream (video) contains another one (subtitles/CC).
CC in e.g. wtv, mxf and even mov as in ticket #846 is actually supported in so far as extracting it is possible, we just can't decode it into plain text or such and we can't mux it either.

[mikecheat04]

I see. Could any of the code from the libzvbi project maybe help with this? It seems they've found a way to gather that information from the video stream and then mux it into a TS. There is also another project called OB-Encoder, they use libav, libx264, and libzvbi with a third party mpegts muxer to produce the desired output. I could use their program but they don't currently support mpeg2 and some of the more advance features of ffmepg that I've accustomed to.

libzvbi -  http://zapping.sourceforge.net/ZVBI/Download.html
obencoder -  http://code.google.com/p/open-broadcast-encoder/downloads/list