[FFmpeg-devel] [RFC] Format of packets for text subtitles

Nicolas George nicolas.george at normalesup.org
Tue Jun 5 22:36:02 CEST 2012


The purpose of this mail is to serve as a template for a specification for
the format of AVPackets for the various text subtitles codecs.

In other words, it aims at defining the duties of the demuxers and muxers
capable of handling text subtitles. Hopefully, if all demuxers and muxers
follow this specification, -scodec copy should work.

The rationale for most suggestions here is: avoid special cases in muxers
and demuxers that are not specifically for text subtitles formats.

(As far as I know, Matroska is by far the most widely used format that
supports various text subtitles styles, and the storage seems pretty sane,
so the previous condition is mostly synonym with "do like in Matroska".)

Clément, you seem to know a lot about subtitles formats, I would appreciate
if you comment on the relevance and correctness of what follow.

Subtitle start time and duration are stored in AVPacket.pts and
AVPacket.duration, in AVStream.time_base units.

(Possible problem: ASS subtitles overlap and can start at the same time. The
Matroska demuxer merges ASS packet with the same start time. OTOH, the text
ASS demuxer does not, so it must not be a really big problem.)

All subtitle packets are flagged as keyframes.

Now for the various codecs:


  AVCodec.extradata empty.

  AVPacket.data contains the bare text (no timestamps), in UTF-8, with line
  separators as either LF or CRLF and no final newline.

  (Problem: OGM often contains text subtitles in legacy encoding without
  encoding specification. I guess a demuxer option should be used.)


  Same as CODEC_ID_TEXT, but the text can contain pseudo-HTML markup.


  AVCodec.extradata contains the declarative sections, down to and including
  the Format specification of the Events section, as plain text. The
  Events/Format specification must be in standard order.

  AVPacket.data contains the text after the "Dialogue:" declaration, except
  the Start and End field, and with a ReadOrder field (an integer, starting
  at 0, that codes the order of the event in the ASS text file, which can be
  not chronological).

Other subtitles formats: I am not familiar enough with the details.

(Big problem: this is a big ABI break, and although it was never documented,
there are certainly programs that rely on the format of ASS packets out of
the Matroska demuxer, starting with MPlayer and VLC. Possible solution: add
an option, enabled by default, to let av_read_frame resynthesize the
timestamp in SRT or ASS syntax.)


  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 198 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20120605/2a2a143b/attachment.asc>

More information about the ffmpeg-devel mailing list