dvb_teletext not detected (ARD HD)

[martii]

Summary of the bug:

In ffmpeg-0.8.12 ffprobe correctly recognizes teletext in transport streams:
Stream #0.2[0x13f0](deu): Subtitle: [6][0][0][0] / 0x0006

In later versions (including the current GIT) the output for the same recorded transport stream is
Stream #0:2[0x13f0](deu): Audio: mp1 ([6][0][0][0] / 0x0006), 0 channels, s16p

I can easily provide a sample on request.

% ./0.8.12/ffprobe ~/test.ts 
ffprobe version 0.8.12, Copyright (c) 2007-2011 the FFmpeg developers
  built on Jan 25 2013 20:48:14 with llvm_gcc 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2335.15.00)
  configuration: --disable-doc
  libavutil    51.  9. 1 / 51.  9. 1
  libavcodec   53.  8. 0 / 53.  8. 0
  libavformat  53.  5. 0 / 53.  5. 0
  libavdevice  53.  1. 1 / 53.  1. 1
  libavfilter   2. 23. 0 /  2. 23. 0
  libswscale    2.  0. 0 /  2.  0. 0
Input #0, mpegts, from '/home/martii/test.ts':
  Duration: 00:00:01.41, start: 50895.134844, bitrate: 5943 kb/s
  Program 28006 
    Stream #0.0[0x13ed]: Video: h264 (High), yuv420p, 1280x720 [PAR 1:1 DAR 16:9], 70 fps, 50 tbr, 90k tbn, 100 tbc
    Stream #0.1[0x13ee](deu): Audio: mp2, 48000 Hz, stereo, s16, 256 kb/s
  Program 10301 
    Stream #0.3[0x492]: Data: [5][0][0][0] / 0x0005
    Stream #0.4[0x87b]: Data: [11][0][0][0] / 0x000B
    Stream #0.5[0x87c]: Data: [11][0][0][0] / 0x000B
    Stream #0.0[0x13ed]: Video: h264 (High), yuv420p, 1280x720 [PAR 1:1 DAR 16:9], 70 fps, 50 tbr, 90k tbn, 100 tbc
    Stream #0.1[0x13ee](deu): Audio: mp2, 48000 Hz, stereo, s16, 256 kb/s
    Stream #0.6[0x13ef](mis): Audio: mp3, 0 channels
    Stream #0.2[0x13f0](deu): Subtitle: [6][0][0][0] / 0x0006
    Stream #0.7[0x13f1](deu): Subtitle: dvbsub (hearing impaired)
    Stream #0.8[0x13f2](deu): Audio: ac3, 0 channels
    Stream #0.9[0x13f4]: Data: [6][0][0][0] / 0x0006
  Program 10302 
  Program 10303 
  Program 10304 
Unsupported codec with id 94215 for input stream 2
Unsupported codec with id 0 for input stream 3
Unsupported codec with id 0 for input stream 4
Unsupported codec with id 0 for input stream 5
Unsupported codec with id 0 for input stream 9

%./git/ffprobe ~/test.ts 
ffprobe version N-49295-ge80be5a0 Copyright (c) 2007-2013 the FFmpeg developers
  built on Jan 25 2013 20:50:17 with llvm-gcc 4.2.1 (LLVM build 2335.15.00)
  configuration: --disable-doc
  libavutil      52. 16.100 / 52. 16.100
  libavcodec     54. 91.100 / 54. 91.100
  libavformat    54. 61.104 / 54. 61.104
  libavdevice    54.  3.102 / 54.  3.102
  libavfilter     3. 33.101 /  3. 33.101
  libswscale      2.  2.100 /  2.  2.100
  libswresample   0. 17.102 /  0. 17.102
Input #0, mpegts, from '/home/martii/test.ts':
  Duration: 00:00:01.41, start: 50895.134844, bitrate: 5943 kb/s
  Program 28006 
  Program 10301 
    Stream #0:3[0x492]: Unknown: none ([5][0][0][0] / 0x0005)
    Stream #0:4[0x87b]: Unknown: none ([11][0][0][0] / 0x000B)
    Stream #0:5[0x87c]: Unknown: none ([11][0][0][0] / 0x000B)
    Stream #0:0[0x13ed]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 50 fps, 50 tbr, 90k tbn, 100 tbc
    Stream #0:1[0x13ee](deu): Audio: mp2 ([3][0][0][0] / 0x0003), 48000 Hz, stereo, s16p, 256 kb/s
    Stream #0:6[0x13ef](mis): Audio: mp3 ([3][0][0][0] / 0x0003), 0 channels
    Stream #0:2[0x13f0](deu): Audio: mp1 ([6][0][0][0] / 0x0006), 0 channels, s16p
    Stream #0:7[0x13f1](deu): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006) (hearing impaired)
    Stream #0:8[0x13f2](deu): Audio: ac3 ([6][0][0][0] / 0x0006), 0 channels
    Stream #0:9[0x13f4]: Unknown: none ([6][0][0][0] / 0x0006)
  Program 10302 
  Program 10303 
  Program 10304 
Unsupported codec with id 0 for input stream 3
Unsupported codec with id 0 for input stream 4
Unsupported codec with id 0 for input stream 5
Unsupported codec with id 0 for input stream 9

[martii]

transport stream sample to illustrate teletext misclassification in ffmpeg > 0.8.12

[cehoyos]

Replying to martii:

In ffmpeg-0.8.12 ffprobe correctly recognizes teletext in transport streams:
Stream #0.2[0x13f0](deu): Subtitle: [6][0][0][0] / 0x0006

No, it recognizes "Subtitle" but not "dvb_teletext" which would allow decoding with third party applications (MPlayer).

Could you provide a very significantly longer sample?
Either upload to  http://www.datafilehost.com/ or read http://ffmpeg.org/bugreports.html

[kierank]

There's no PAT (and probably no PMT) present in the sample so you can't probe properly. Cut a larger sample if possible.

[martii]

Thanks, I've uploaded a longer transport stream to

 http://www.datafilehost.com/download-b1107303.html

This is at nearly 60MB, about 80 seconds, SD quality.

Delete link:  http://www.datafilehost.com/delete-b1107303-10822366.html

0.8.12: Stream #0.2[0x68](deu): Subtitle: [6][0][0][0] / 0x0006
current: Stream #0:2[0x68](deu): Audio: mp1 ([6][0][0][0] / 0x0006), 0 channels, s16p

Thanks!

[cehoyos]

I would like to suggest that you post a download link without a delete link;-)

[martii]

The delete link was on purpose. I'm pretty sure I would have lost that link, and there shouldn't be any need to keep that file longer in place than necessary. In case it was deleted by some kid just for fun before you were able to retrieve it, here's the same file again:

 http://www.datafilehost.com/download-e84e93eb.html

Thank you!

[martii]

Is anyone caring about this issue, or is it in some "ignore, won't fix" state?

[cehoyos]

Can you explain how you recorded the sample?

The reason I ask is that I can record ARD here with mplayer dvb://ARD -dumpstream and the teletext stream is correctly detected by ffmpeg -i stream.dump (will upload as ARD-UPC.ts), so I wonder if you are simply recording the stream without the teletext data (or, as Kieran already assumed, more likely the PMT).

(I originally did not react anymore because you completely changed the stream that you uploaded which made the ticket somehow invalid imo, but please don't mind that.)

[martii]

Thanks a lot for the feedback. The recordings were made using Neutrino running on a satellite receiver. The actual Neutrino version is from  https://gitorious.org/~martii/neutrino-mp/martiis-neutrino-mp (which is "my" GIT, and in the process of checking what could go wrong on my side I indeed found a couple of recently introduced bugs, but these are unrelated to the issue I'm seeing with ffmpeg).

I've verified that the code records both PAT and PMT, and I've even done a quick analysis with dvbsnoop that confirms that PAT and PMT packets are being written to file. Recording is done directly from the demux device, so the stream should be pretty much indentical to what's sent via satellite.

VLC easily recognizes (and plays) all the streams correctly, including the Teletext one.

[cehoyos]

Can you test mplayer -dumpstream on your setup?

[martii]

Not trivially ... I assume you want me to access /dev/dvb/adapter0/demux0 or /dev/dvb/adapter0/dvr0 directly? Alas, the receiver is a sh4 architecture, and I don't have a mplayer binary for that. I guess I could try to cross-compile one, but that might take some time, and I don't believe that the output streams would differ from what I'm seeing now when recording or streaming to the network (both functions are just read-write loops).

I'll try to get mplayer up and running on that box.

[cehoyos]

It unfortunately may take some time until I have access to my DVB-S(2) receiver to test myself.
(I assume you are recording from Astra, aren't you?)

Did you test the sample I uploaded? Do you see any obvious differences to your sample with the tools you use?

[martii]

Yes, this is Astra 19.2E. I've cross-compiled mplayer and can record video and audio; however, I haven't figured out how to add arbitrary PIDs yet. The channels.conf seems to be unsuitable for that.

I've had a look at your sample. It contains several tables which I'm currently ignoring (SDT, EIT, others) but I'm fairly sure that I shouldn't need those. I will give this another shot later this week; it's already late and I have to get up early in the morning.

Thanks!

[cehoyos]

Replying to martii:

I've cross-compiled mplayer and can record video and audio; however, I haven't figured out how to add arbitrary PIDs yet. The channels.conf seems to be unsuitable for that.

See the line I use for an example:
ARD:314000000:INVERSION_AUTO:6900000:FEC_AUTO:QAM_256:2000+2001+2101+2071:2041+2011+2012:2001
Iirc, 2000 is the pmt, 2001 video, 2101 contains teletext and 2071 the dvb subtitles, remaining ones are audio.

[martii]

This is pretty embarrassing. Sorry for wasting your time. Please mark this ticket as "invalid" or whatever seems appropriate.

Upon closer inspection the Neutrino code writes custom-build PAT and PMT packets at the beginning of a recorded file. These initial, hand-crafted packets (which should purportedly make the recorded stream playable under Enigma2) might speed up probing, but apparently don't come with subtitle or teletext pids. The streams work fine with ffmpeg if these fake PAT/PMT packets aren't added.

Thanks a lot for your help!

[cehoyos]

Thank you for the clarification!