[FFmpeg-devel] Captions SCC

[Devin Heitmueller]

On Fri, Feb 7, 2025 at 1:32 PM Devlist Archive <devlist at rlb.org> wrote:
> That is good to hear, FYI any time 708 captions present in a DTVCC video
> stream there also must be 608 captions wrapped in that stream for legal
> compliance and backwards compatibility with older downstream equipment
> that downscales to SD.  Typically files come in with both 608 and 708
> streams, but sometimes there will only be a 608 stream.  Regulations only
> require 608, but 708 support is ideal.

Yeah, almost everybody ignores the 708 data, and in fact in most cases
the content is authored in 608 and then upconverted to 708 (i.e. so
you don't get any of the benefits of 708).  It's unfortunate how that
turned out, but it's largely related to economics for the content
providers.

I considered adding a c708 AVPacket format which preserves the 708
data, but never got around to it.  Even the MCC demuxer in ffmpeg
right now only uses the 608 tuples and throws away the 708 data.  It's
kind of a mess.

> I noticed that either the RCWT or ccextractor scc process is also messing
> with the display duration of the captions, so that is not a currently
> functional workflow for me.  Probably has something to do with the
> processing subcc is doing.  At least that workflow makes a more accurate
> scc file than what is produced when using ffmpeg's ability to write the
> .scc file directly.
>
> The functions on my wish list are full 708/608 support for extraction and
> embedding to/from mcc and preserving 708/608 during frame rate conversion
> and transcoding with ffmpeg.  I have been pleased to see that transcoding
> now works with standard commands, but frame rate conversion using normal
> commands does not automatically invoke the cc_fifo mechanism or cc_repack.
> It would seem that captions should be handled correctly automatically and
> not need special filter commands to preserve them.  The captions I work
> with are basically user data captions but in an mp4 file according to SCTE
> 128 / DTVCC Transport.

The cc_fifo mechanism is actually implemented within the various
filters which change the framerate (deinterlacing, framerate
conversion, interlacing), so no extra command line arguments are
required.  There are some edge cases though that don't properly handle
it, and those mainly relate to cases such as the captions crossing
back/forth between SEI and dedicated caption tracks (e.g. MP4 with a
caption track, the subcc output, etc).

The cc_repack filter itself should almost never be actually used.
It's there mainly to deal with certain broken streams to regenerate
the padding in a conformant manner (e.g. deals with cases where an
incorrect amount of padding is in the source stream, the 608 tuples
violate the interleaving rules, etc).

Devin

-- 
Devin Heitmueller, Senior Software Engineer
LTN Global Communications
o: +1 (301) 363-1001
w: https://ltnglobal.com  e: devin.heitmueller at ltnglobal.com