support for copying SMPTE 436M caption tracks

[dave rice]

This is a part 2 to ​http://ffmpeg.org/trac/ffmpeg/ticket/682.

I'm working with media from XDCam discs which contain an MXF with mpeg2 video, 8 pcm audio tracks, and a caption track using SMPTE 436M. I just need to cut these files given an in point and duration and copy all tracks to a new file (using -map 0 -c copy).

For XDCam MXF files the caption track is not recognized:

[mxf @ 0x101893800] track 9: could not find essence container ul, codec not currently supported in container

and subsequently the copy fails with

Could not write header for output file #0 (incorrect codec parameters ?)

I need the preserve the captions but would also like to use ffmpeg to this task.

Full output (first 5MB of a sample attached):

ffmpeg -ss 300 -t 30 -i ELCO0378.mxf -map 0 -c copy trimmed.mxf
ffmpeg version N-31468-g31c55c5, Copyright (c) 2000-2011 the FFmpeg developers
  built on Dec  8 2011 08:46:52 with gcc 4.2.1 (Apple Inc. build 5666) (dot 3)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-gpl --enable-version3 --enable-nonfree --enable-hardcoded-tables --cc=/usr/bin/gcc-4.2 --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libxvid --disable-ffplay
  libavutil    51. 30. 0 / 51. 30. 0
  libavcodec   53. 41. 2 / 53. 41. 2
  libavformat  53. 24. 0 / 53. 24. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 51. 0 /  2. 51. 0
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[mxf @ 0x10180f200] could not resolve sub descriptor strong ref
[mxf @ 0x10180f200] source track 11: stream 9, no descriptor found
Input #0, mxf, from 'ELCO0378.mxf':
  Duration: 00:28:50.89, start: 0.000000, bitrate: 60317 kb/s
    Stream #0:0: Video: mpeg2video (4:2:2), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], 50000 kb/s, 29.97 fps, 29.97 tbr, 29.97 tbn, 59.94 tbc
    Stream #0:1: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:2: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:3: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:4: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:5: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:6: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:7: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:8: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:9: Data: none
[mxf @ 0x101893800] track 9: could not find essence container ul, codec not currently supported in container
Output #0, mxf, to 'trimmed.mxf':
  Metadata:
    encoder         : Lavf53.24.0
    Stream #0:0: Video: mpeg2video, yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 50000 kb/s, 29.97 fps, 29.97 tbn, 29.97 tbc
    Stream #0:1: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:2: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:3: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:4: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:5: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:6: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:7: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:8: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:9: Data: none
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (copy)
  Stream #0:3 -> #0:3 (copy)
  Stream #0:4 -> #0:4 (copy)
  Stream #0:5 -> #0:5 (copy)
  Stream #0:6 -> #0:6 (copy)
  Stream #0:7 -> #0:7 (copy)
  Stream #0:8 -> #0:8 (copy)
  Stream #0:9 -> #0:9 (copy)
Could not write header for output file #0 (incorrect codec parameters ?)

[dave rice]

first 2.5 MB of an MXF file with SMPTE 436M caption track

[Carl Eugen Hoyos]

Isn't this first of all a request to be able to read the caption track?

[dave rice]

Sure. Reading captions via ffmpeg would be ideal. I assume that having a smpte 436m encoder/decoder would be a lot of work. At this time I only have a use case for copying 436m captions from one MXF to another MXF, which I hope is a simpler request. :)

[reimar]

To support this we would at least have to know what format that is, to at least give it the correct name.
Your sample does not contain enough data to figure out much, except that it is not the VBI format that e.g. MXFLib supports.

[dave rice]

extract of smpte 436M vbi caption track extracted from an XDCam disc's MXF with mxfsplit

[dave rice]

I realized that the MXF sample I sent was too short to contain the caption track. I just attached an extract of the caption track extracted with mxfsplit. You'll see the textual part of the caption start at 0x46D9E. This sample uses 92 bytes per frame.

[dave rice]

I uploaded a full sample file to ​http://www.archive.org/download/xdcam_sample_with_caption_track/C0001.MXF.

ffmpeg started on 2011-12-20 at 11:40:58
Report written to "ffmpeg-20111220-114058.log"
Command line:
ffmpeg -y -report -ss 50 -t 20 -i C0001.MXF -map 0 -c copy C0001_trim.mxf
ffmpeg version N-32346-g62b5197, Copyright (c) 2000-2011 the FFmpeg developers
  built on Dec 15 2011 11:51:15 with gcc 4.2.1 (Apple Inc. build 5666) (dot 3)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/HEAD --enable-shared --enable-gpl --enable-version3 --enable-libfreetype --enable-nonfree --enable-hardcoded-tables --cc=/usr/bin/gcc-4.2 --enable-libx264 --enable-libfaac --enable-libmp3lame --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libxvid
  libavutil    51. 32. 0 / 51. 32. 0
  libavcodec   53. 45. 0 / 53. 45. 0
  libavformat  53. 25. 0 / 53. 25. 0
  libavdevice  53.  4. 0 / 53.  4. 0
  libavfilter   2. 53. 0 /  2. 53. 0
  libswscale    2.  1. 0 /  2.  1. 0
  libpostproc  51.  2. 0 / 51.  2. 0
[mxf @ 0x7faed3049800] Format mxf probed with size=2048 and score=100
[mxf @ 0x7faed3049800] could not resolve sub descriptor strong ref
[mxf @ 0x7faed3049800] source track 11: stream 9, no descriptor found
[mpeg2video @ 0x7faed3077000] err{or,}_recognition separate: 1; 1
[mpeg2video @ 0x7faed3077000] err{or,}_recognition combined: 1; 10001
[pcm_s24le @ 0x7faed304fa00] err{or,}_recognition separate: 1; 1
[pcm_s24le @ 0x7faed304fa00] err{or,}_recognition combined: 1; 10001
[pcm_s24le @ 0x7faed3096200] err{or,}_recognition separate: 1; 1
[pcm_s24le @ 0x7faed3096200] err{or,}_recognition combined: 1; 10001
[pcm_s24le @ 0x7faed3050000] err{or,}_recognition separate: 1; 1
[pcm_s24le @ 0x7faed3050000] err{or,}_recognition combined: 1; 10001
[pcm_s24le @ 0x7faed30aec00] err{or,}_recognition separate: 1; 1
[pcm_s24le @ 0x7faed30aec00] err{or,}_recognition combined: 1; 10001
[pcm_s24le @ 0x7faed3050600] err{or,}_recognition separate: 1; 1
[pcm_s24le @ 0x7faed3050600] err{or,}_recognition combined: 1; 10001
[pcm_s24le @ 0x7faed3050c00] err{or,}_recognition separate: 1; 1
[pcm_s24le @ 0x7faed3050c00] err{or,}_recognition combined: 1; 10001
[pcm_s24le @ 0x7faed3051200] err{or,}_recognition separate: 1; 1
[pcm_s24le @ 0x7faed3051200] err{or,}_recognition combined: 1; 10001
[pcm_s24le @ 0x7faed3051800] err{or,}_recognition separate: 1; 1
[pcm_s24le @ 0x7faed3051800] err{or,}_recognition combined: 1; 10001
[mpeg2video @ 0x7faed3077000] Unsupported bit depth: 0

Seems stream 0 codec frame rate differs from container frame rate: 59.94 (60000/1001) -> 29.97 (60000/2002)
Input #0, mxf, from 'C0001.MXF':
  Duration: 00:01:10.27, start: 0.000000, bitrate: 60357 kb/s
    Stream #0:0, 1, 1001/30000: Video: mpeg2video (4:2:2), yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], 50000 kb/s, 29.97 tbr, 29.97 tbn, 59.94 tbc
    Stream #0:1, 1, 1001/30000: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:2, 1, 1001/30000: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:3, 1, 1001/30000: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:4, 1, 1001/30000: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:5, 1, 1001/30000: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:6, 1, 1001/30000: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:7, 1, 1001/30000: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:8, 1, 1001/30000: Audio: pcm_s24le, 48000 Hz, 1 channels, s32, 1152 kb/s
    Stream #0:9, 0, 1001/30000: Data: none
[mxf @ 0x7faed30c7c00] track 9: could not find essence container ul, codec not currently supported in container
Output #0, mxf, to 'C0001_trim.mxf':
  Metadata:
    encoder         : Lavf53.25.0
    Stream #0:0, 0, 1001/30000: Video: mpeg2video, yuv422p, 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 50000 kb/s, 29.97 tbn, 29.97 tbc
    Stream #0:1, 0, 1/48000: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:2, 0, 1/48000: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:3, 0, 1/48000: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:4, 0, 1/48000: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:5, 0, 1/48000: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:6, 0, 1/48000: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:7, 0, 1/48000: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:8, 0, 1/48000: Audio: pcm_s24le, 48000 Hz, 1 channels, 1152 kb/s
    Stream #0:9, 0, 1/90000: Data: none
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
  Stream #0:1 -> #0:1 (copy)
  Stream #0:2 -> #0:2 (copy)
  Stream #0:3 -> #0:3 (copy)
  Stream #0:4 -> #0:4 (copy)
  Stream #0:5 -> #0:5 (copy)
  Stream #0:6 -> #0:6 (copy)
  Stream #0:7 -> #0:7 (copy)
  Stream #0:8 -> #0:8 (copy)
  Stream #0:9 -> #0:9 (copy)
Could not write header for output file #0 (incorrect codec parameters ?)
 

[helmboy]

To support any VBI data in MXF the UUID for ancillary essences would have to be added (060E2B34-04010109-0D010301-020E0000). Each of these essence packets begins with a 16 byte header that specifies the SDI VANC field, line, packet wrapping, YUV sample coding and data counts.

Then the 8-bit DID, SDID and DC follow.

The DID for EIA608 and EIA-708 is 97 with the SDID of 1 for 708 and 2 for 608

Then for a EIA-708 payload, this is encapsulated in a Caption Distribution Packet, which always has EIA-608 captions as well.

The Caption Distribution Packet id is 0x9669

Fully structure details are on Wikipedia:

​http://en.wikipedia.org/wiki/CEA-708

and

​https://en.wikipedia.org/wiki/EIA-608

I hope this is useful to any one.

[helmboy]

In terms of parsing the MXF demuxer should pass all the data after the DC to the codec which would handle the CDP and override the timestamps when the internal timecode is set. The codec would be selected by the DID and SDID. The ancillary header always uses the same field of 1, line of 9, wrapping of 1 and YUV coding of 4.

I would write the code myself, but lack enough MXF source files to test it on.

[Rafaël Carré]

Here is the 16 bytes header before CDP:
(bytes) - desc
(2) - Number of ANC packets
(2) - VANC Line number
(1) - Wrapping type (bit 4 is HANC if set, VANC if not).

bits 2-0:

1 == interlaced or spf frame
2 == field 1
3 == field 2
4 == progressive frame

(1) - payload sample config

4 == 8 bit luma
5 == 8 bit color difference
6 == 8 bit luma & color difference
7 == 10 bit luma
8 == 10 bit color difference
9 == 10 bit luma & color difference

10 == 8 bit luma with parity
11 == 8 bit color difference with parity
12 == 8 bit luma & color difference with parity

(2) - payload sample count (bytes following 16 bytes header)
(8) - ???

For the attached sample the last 8 bytes are 0000 004c 0000 0001, for another sample I have it's 0000 0050 0000 0001
It could be some variation of the "ADF" Ancillary Data Flag which is 0x000 0x3ff 0x3ff

[Rafaël Carré]

Here is an implementation: ​https://github.com/Tjoppen/mxflib/blob/master/mxflib/vbi.cpp

[da8eat]

I explained last 8 bytes in the post by the link below
​http://mogurenko.com/2015/10/30/closed-captions-and-mxf/

Replying to funman:

(8) - ???

For the attached sample the last 8 bytes are 0000 004c 0000 0001, for another sample I have it's 0000 0050 0000 0001
It could be some variation of the "ADF" Ancillary Data Flag which is 0x000 0x3ff 0x3ff

[Vince]

Will this change allow any vbi_vanc_smpte_436m stream to be passed on processing of the .mxf file.
Or should I raise a RFE such that any "vbi_vanc_smpte_436m" stream in an mxf file be copied. I'm not looking for the content of the stream to be decoded but just copied in the same way you would use "-c:v copy -c:a copy" for video and audio?.

[Vince]

Replying to cehoyos:

Isn't this first of all a request to be able to read the caption track?

No it isn't, its a request to just copy the data tracks, not decode them nor encode them just copy them.

[Elon Musk]

Have not this bug been fixed, I see code in mxf muxer for smpte 436.

[dave rice]

Yes S436m tracks now seem to copy and the commands I originally wrote appear to now work in ffmpeg. I'm okay with closing this as resolved.