Muxing dvb_teletext in mpegts does not work

[burek]

Summary of the bug:
When ffmpeg is instructed to do "-c copy" on input stream, it unexpectedly converts one of the input streams of type dvb_teletext into dvb_subtitle. More info can be found here:
 http://ffmpeg.gusari.org/viewtopic.php?f=16&t=811&p=1527

How to reproduce:
The original scenario, where the possible bug was discovered, used a command line like this:

ffmpeg -i udp://234.5.245.105:20000 -map 0 -codec copy -f mpegts stream.ts

ffmpeg version 1.1.1 Copyright (c) 2000-2013 the FFmpeg developers
  built on Jan 21 2013 14:57:54 with gcc 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)
  configuration: --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 --enable-nonfree --enable-postproc --enable-version3 --enable-x11grab --enable-filter=movie --enable-static --enable-shared --enable-libfreetype --enable-libaacplus
  libavutil      52. 13.100 / 52. 13.100
  libavcodec     54. 86.100 / 54. 86.100
  libavformat    54. 59.106 / 54. 59.106
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 32.100 /  3. 32.100
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0x1b5c820] Unable to seek back to the start
[mpeg2video @ 0x1ba2220] 0x0 is invalid
    Last message repeated 15 times
[mpegts @ 0x1b5c820] max_analyze_duration 5000000 reached at 5016000
[mpegts @ 0x1b5c820] Estimating duration from bitrate, this may be inaccurate
Input #0, mpegts, from 'udp://234.5.245.105:20000':
  Duration: N/A, start: 93705.929667, bitrate: 224 kb/s
  Program 140 
    Metadata:
      service_name    : ANTENA 3
      service_provider: EtherTV
    Stream #0:0[0x137](spa): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
    Stream #0:1[0x130](und): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 96 kb/s
    Stream #0:2[0x12f](spa): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 128 kb/s
    Stream #0:3[0x12e](spa): [b]Subtitle: dvb_teletext[/b] ([6][0][0][0] / 0x0006)
    Stream #0:4[0x12d]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 704x576 [SAR 16:11 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
Output #0, mpegts, to 'stream.ts':
  Metadata:
    encoder         : Lavf54.59.106
    Stream #0:0(spa): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
    Stream #0:1(und): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, 96 kb/s
    Stream #0:2(spa): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, 128 kb/s
    Stream #0:3(spa): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
    Stream #0:4: Video: mpeg2video ([2][0][0][0] / 0x0002), yuv420p, 704x576 [SAR 16:11 DAR 16:9], q=2-31, 25 fps, 90k tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (copy)
  Stream #0:3 -> #0:3 (copy)
  Stream #0:4 -> #0:4 (copy)
Press [q] to stop, [?] for help
frame=  914 fps= 29 q=-1.0 Lsize=   17315kB time=00:00:37.79 bitrate=3753.4kbits/s    
video:13770kB audio:1013kB subtitle:1173 global headers:0kB muxing overhead 8.519125%
Received signal 2: terminating.

but when checked with ffprobe, the result was:

ffprobe -i stream.ts 

ffprobe version 1.1.1 Copyright (c) 2007-2013 the FFmpeg developers
  built on Jan 21 2013 14:57:54 with gcc 4.6.1 (Ubuntu/Linaro 4.6.1-9ubuntu3)
  configuration: --enable-gpl --enable-libfaac --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libtheora --enable-libvorbis --enable-libx264 --enable-nonfree --enable-postproc --enable-version3 --enable-x11grab --enable-filter=movie --enable-static --enable-shared --enable-libfreetype --enable-libaacplus
  libavutil      52. 13.100 / 52. 13.100
  libavcodec     54. 86.100 / 54. 86.100
  libavformat    54. 59.106 / 54. 59.106
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 32.100 /  3. 32.100
  libswscale      2.  1.103 /  2.  1.103
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[mpegts @ 0xac2ca0] max_analyze_duration 5000000 reached at 5016000
[NULL @ 0xac6e60] start time is not set in estimate_timings_from_pts
[NULL @ 0xac8be0] start time is not set in estimate_timings_from_pts
Input #0, mpegts, from 'stream.ts':
  Duration: 00:00:37.47, start: 1.760378, bitrate: 3785 kb/s
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100](spa): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
    Stream #0:1[0x101](und): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 96 kb/s
    Stream #0:2[0x102](spa): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 128 kb/s
    Stream #0:3[0x103](spa): [b]Subtitle: dvb_subtitle[/b] ([6][0][0][0] / 0x0006)
    Stream #0:4[0x104]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p, 704x576 [SAR 16:11 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc

[cehoyos]

Is this only reproducible with a network input stream or also with file input?

[Cigaes]

AFAICS, this is reproducible with the sample you uploaded for ticket 2086  http://samples.ffmpeg.org/ffmpeg-bugs/trac/ticket2086/.

Input #0, mpegts, from '/tmp/teletextsubtitles.ts.part':
  Duration: 00:00:15.36, start: 0.091256, bitrate: 15069 kb/s
  Program 137 
    Stream #0:0[0x30]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x40](ita): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Stream #0:2[0x41](und): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Stream #0:3[0x6a](ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
Output #0, mpegts, to '/tmp/out.ts':
  Metadata:
    encoder         : Lavf54.61.104
    Stream #0:0(ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
Stream mapping:
  Stream #0:3 -> #0:0 (copy)

Input #0, mpegts, from '/tmp/out.ts':
  Duration: N/A, bitrate: N/A
  Program 1 
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100](ita): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)

I suspect this is a muxer problem: ffmpeg does not recode the subtitle packets, it only incorrectly writes the nature of the stream.

[cehoyos]

Attached patch does not really help.

$ ffmpeg -i teletextsubtitles.ts -vn -an -scodec copy out.ts
ffmpeg version N-49553-g13eb9fc Copyright (c) 2000-2013 the FFmpeg developers
  built on Feb  3 2013 14:58:47 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 17.100 / 52. 17.100
  libavcodec     54. 91.100 / 54. 91.100
  libavformat    54. 61.104 / 54. 61.104
  libavdevice    54.  3.103 / 54.  3.103
  libavfilter     3. 35.100 /  3. 35.100
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[h264 @ 0x286dee0] non-existing SPS 0 referenced in buffering period
[h264 @ 0x286dee0] non-existing PPS referenced
[h264 @ 0x286dee0] non-existing SPS 0 referenced in buffering period
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] no frame!

...

[h264 @ 0x286dee0] non-existing SPS 0 referenced in buffering period
[h264 @ 0x286dee0] non-existing PPS referenced
[h264 @ 0x286dee0] non-existing SPS 0 referenced in buffering period
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] non-existing PPS 0 referenced
[h264 @ 0x286dee0] decode_slice_header error
[h264 @ 0x286dee0] no frame!
[mpegts @ 0x2869fc0] Stream #3: not enough frames to estimate rate; consider increasing probesize
[NULL @ 0x286fc60] start time is not set in estimate_timings_from_pts
[mpegts @ 0x2869fc0] PES packet size mismatch
    Last message repeated 1 times
Input #0, mpegts, from 'teletextsubtitles.ts':
  Duration: 00:00:55.32, start: 0.091256, bitrate: 14610 kb/s
  Program 137
    Stream #0:0[0x30]: Video: h264 (Main) ([27][0][0][0] / 0x001B), yuv420p, 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 90k tbn, 50 tbc
    Stream #0:1[0x40](ita): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Stream #0:2[0x41](und): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
    Stream #0:3[0x6a](ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
Output #0, mpegts, to 'out.ts':
  Metadata:
    encoder         : Lavf54.61.104
    Stream #0:0(ita): Subtitle: dvb_teletext ([6][0][0][0] / 0x0006)
Stream mapping:
  Stream #0:3 -> #0:0 (copy)
Press [q] to stop, [?] for help
size=     264kB time=00:00:00.00 bitrate=N/A
video:0kB audio:0kB subtitle:185 global headers:0kB muxing overhead 42.903811%

$ ffmpeg -i out.ts
ffmpeg version N-49553-g13eb9fc Copyright (c) 2000-2013 the FFmpeg developers
  built on Feb  3 2013 14:58:47 with gcc 4.7 (SUSE Linux)
  configuration: --enable-gpl --disable-indev=jack
  libavutil      52. 17.100 / 52. 17.100
  libavcodec     54. 91.100 / 54. 91.100
  libavformat    54. 61.104 / 54. 61.104
  libavdevice    54.  3.103 / 54.  3.103
  libavfilter     3. 35.100 /  3. 35.100
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
  libpostproc    52.  2.100 / 52.  2.100
[NULL @ 0x1e98720] start time is not set in estimate_timings_from_pts
Input #0, mpegts, from 'out.ts':
  Duration: N/A, bitrate: N/A
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
    Stream #0:0[0x100](ita): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
At least one output file must be specified

[cehoyos]

Please tell the "true" OP that mplayer -dumpstream is the recommended way of recording a transport stream from dvb (imo), it allows to save all wanted streams / pids.

[cehoyos]

To the original reporter:
There is a mailing list where user questions are answered: http://ffmpeg.org/mailman/listinfo/ffmpeg-user
Concerning your problem: To dump transport streams, do not use FFmpeg's remuxing function (it is not made for this task), instead use (for example) mplayer -dumpstream. (I am sure there are other possibilities, this is how I do it.)

In case my comment:3 was not clear: The attached patch does *not* help, it does *not* fix your issue.

[milan2655]

I added two files with component section of PMT for teletext. One is from original stream (PMT - teletext OK.txt) and second one (PMT - teletext NOT OK.txt) is from stream that is generated with ffmpeg with patchdvbteletext applied.

The patchdvbteletext added two additional teletext_list entries with strange ISO_639_language_code.

Teletext generated by ffmpeg with patchdvbteletext applied cannot be viewed on any device.

I'm using DVB inspector as teletext viewer.

[cehoyos]

I deleted the attachment, sorry that it caused so much confusion, I will try to avoid attaching wip in the future.

[milan2655]

You were on good way of solving this problem. Who and how will solve teletex problem ?

Your patch was OK. It changed descriptor from 59 to 56 for teletext, but added additional stuff.

[cehoyos]

Replying to milan2655:

Who and how will solve teletex problem ?

Patch definitely welcome!

[dnk]

I attached patch which solve the teletext problem.

For testting I used this command:

$ ffmpeg -i ./teletextsubtitles.ts -map 0 -codec copy -f mpegts out.ts

[cehoyos]

Thank you for working on this, please send the patch to ffmpeg-devel where it will receive more attention.