[FFmpeg-devel] [PATCH v5] Add SUP/PGS subtitle demuxer

wm4 nfxjfg at googlemail.com
Mon Sep 1 16:40:39 CEST 2014


On Mon, 01 Sep 2014 13:37:23 +0300
Petri Hintukainen <phintuka at gmail.com> wrote:

> On ma, 2014-09-01 at 10:05 +0200, Hendrik Leppkes wrote:
> > On Mon, Sep 1, 2014 at 10:01 AM, Petri Hintukainen <phintuka at gmail.com> wrote:
> > > On ma, 2014-09-01 at 00:34 +0200, wm4 wrote:
> > >> ---
> > >> Use AV_RB16 instead of memcpy.
> > >> Don't use AVPROBE_SCORE_EXTENSION.
> > >> ---
> > >>  libavformat/Makefile     |   1 +
> > >>  libavformat/allformats.c |   1 +
> > >>  libavformat/supdec.c     | 107 +++++++++++++++++++++++++++++++++++++++++++++++
> > >>  3 files changed, 109 insertions(+)
> > >>  create mode 100644 libavformat/supdec.c
> > >>
> > >> diff --git a/libavformat/Makefile b/libavformat/Makefile
> > >> index 3d124fb..b4965fe 100644
> > >> --- a/libavformat/Makefile
> > >> +++ b/libavformat/Makefile
> > >> @@ -405,6 +405,7 @@ OBJS-$(CONFIG_SRT_MUXER)                 += srtenc.o
> > >>  OBJS-$(CONFIG_STR_DEMUXER)               += psxstr.o
> > >>  OBJS-$(CONFIG_SUBVIEWER1_DEMUXER)        += subviewer1dec.o subtitles.o
> > >>  OBJS-$(CONFIG_SUBVIEWER_DEMUXER)         += subviewerdec.o subtitles.o
> > >> +OBJS-$(CONFIG_SUP_DEMUXER)               += supdec.o
> > >>  OBJS-$(CONFIG_SWF_DEMUXER)               += swfdec.o swf.o
> > >>  OBJS-$(CONFIG_SWF_MUXER)                 += swfenc.o swf.o
> > >>  OBJS-$(CONFIG_TAK_DEMUXER)               += takdec.o apetag.o img2.o rawdec.o
> > >> diff --git a/libavformat/allformats.c b/libavformat/allformats.c
> > >> index 8f70c4b..e6c0e5f 100644
> > >> --- a/libavformat/allformats.c
> > >> +++ b/libavformat/allformats.c
> > >> @@ -280,6 +280,7 @@ void av_register_all(void)
> > >>      REGISTER_DEMUXER (STR,              str);
> > >>      REGISTER_DEMUXER (SUBVIEWER1,       subviewer1);
> > >>      REGISTER_DEMUXER (SUBVIEWER,        subviewer);
> > >> +    REGISTER_DEMUXER (SUP,              sup);
> > >>      REGISTER_MUXDEMUX(SWF,              swf);
> > >>      REGISTER_DEMUXER (TAK,              tak);
> > >>      REGISTER_MUXER   (TEE,              tee);
> > >> diff --git a/libavformat/supdec.c b/libavformat/supdec.c
> > >> new file mode 100644
> > >> index 0000000..3726c97
> > >> --- /dev/null
> > >> +++ b/libavformat/supdec.c
> > >> @@ -0,0 +1,107 @@
> > >> +/*
> > >> + * This file is part of FFmpeg.
> > >> + *
> > >> + * FFmpeg is free software; you can redistribute it and/or
> > >> + * modify it under the terms of the GNU Lesser General Public
> > >> + * License as published by the Free Software Foundation; either
> > >> + * version 2.1 of the License, or (at your option) any later version.
> > >> + *
> > >> + * FFmpeg is distributed in the hope that it will be useful,
> > >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> > >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> > >> + * Lesser General Public License for more details.
> > >> + *
> > >> + * You should have received a copy of the GNU Lesser General Public
> > >> + * License along with FFmpeg; if not, write to the Free Software
> > >> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> > >> + */
> > >> +
> > >> +#include "avformat.h"
> > >> +#include "internal.h"
> > >> +#include "libavutil/intreadwrite.h"
> > >> +
> > >> +#define SUP_PGS_MAGIC 0x5047 /* "PG", big endian */
> > >> +
> > >> +static int sup_read_header(AVFormatContext *s)
> > >> +{
> > >> +    AVStream *st = avformat_new_stream(s, NULL);
> > >> +    if (!st)
> > >> +        return AVERROR(ENOMEM);
> > >> +    st->codec->codec_type = AVMEDIA_TYPE_SUBTITLE;
> > >> +    st->codec->codec_id = AV_CODEC_ID_HDMV_PGS_SUBTITLE;
> > >> +    avpriv_set_pts_info(st, 32, 1, 90000);
> > >> +
> > >> +    return 0;
> > >> +}
> > >> +
> > >> +static int sup_read_packet(struct AVFormatContext *s, AVPacket *pkt)
> > >> +{
> > >> +    int64_t pts, pos;
> > >> +    int ret;
> > >> +
> > >> +    pos = avio_tell(s->pb);
> > >> +
> > >> +    if (avio_rb16(s->pb) != SUP_PGS_MAGIC)
> > >> +        return avio_feof(s->pb) ? AVERROR_EOF : AVERROR_INVALIDDATA;
> > >> +
> > >> +    pts = avio_rb32(s->pb);
> > >> +    avio_rb32(s->pb); /* discard DTS (usually 0, and useless) */
> > >
> > > Is there some reason to discard DTS ? I would keep DTS when it is
> > > present, even if ffmpeg decoder does not use it. There are several
> > > reasons why DTS should be kept:
> > >
> > 
> >  Setting a DTS of 0 for all frames could result in serious problems.
> 
> Unless it is set only when > 0 ?

Can we assume this?

Assuming you never can have the same DTS values, you could read 2
packets at start. If both have DTS=0, you know that DTS should be
discarded. But that sounds like additional effort. Is there a way to
make libavformat/utils.c do this?

All in all, it might just be best to discard DTS if it's 0.

Suggestions welcome.

Yes, we can't pass through DTS if it's all set to 0, it would break too
much.

> >  Shouldn't DTS and PTS be the same for subtitles anyway,
> 
> The stream does not carry subtitles, it carries PGS (Presentation
> Graphics Stream) segments. Each segment has separate PTS/DTS timestamps.
> Subtitle is a result of applying composition rules from two or more
> Presentation Composition Segments to the data carried in other
> segments. 
> 
> > considering they are not re-ordered?
> 
> In PGS stream DTS is monotonic but PTS is not. Segments are (kind of)
> re-ordered after decoding. Here are timestamps of the segments used to
> compose two first subtitles in #2208 sample (00038.m2ts):
> 
> First subtitle:
> SEG 0x16 DTS 000249146 PTS 000255255  // PCS
> SEG 0x17 DTS 000249146 PTS 000254978  // WDS
> SEG 0x14 DTS 000249146 PTS 000249146  // PDS
> SEG 0x15 DTS 000249146 PTS 000249699  // ODS
> SEG 0x80 DTS 000249699 PTS 000249699  // EOD
> 
> Hide first subtitle:
> SEG 0x16 DTS 000607829 PTS 000608107  // PCS
> SEG 0x17 DTS 000607829 PTS 000607830  // WDS
> SEG 0x80 DTS 000607829 PTS 000607829  // EOD
> 
> Second subtitle:
> SEG 0x16 DTS 000609547 PTS 000615615  // PCS
> SEG 0x17 DTS 000609547 PTS 000615379  // WDS
> SEG 0x14 DTS 000609547 PTS 000609547  // PDS
> SEG 0x15 DTS 000609547 PTS 000610019  // ODS
> SEG 0x80 DTS 000610019 PTS 000610019  // EOD
> 
> Hide second subtitle:
> SEG 0x16 DTS 000743005 PTS 000743242  // PCS
> SEG 0x17 DTS 000743005 PTS 000743006  // WDS
> SEG 0x80 DTS 000743005 PTS 000743005  // EOD
> 
> Each segment has separate timestamps. PTS of Presentation Composition
> Segment is the time when subtitle is displayed or hidden. PTS in
> palette / object segments tells when the segment is valid (and available
> to be used in compositing).
> First segment in the stream is presentation composition segment.
> Following window, palette, object... segments must have smaller PTS or
> those could not be used when compositing the image.
> 
> The format allows quite complex animated presentation graphics at full
> HD resolution. Muxing the stream based on PTS only could be fatal for
> resource-limited HW player. DTS can also be important to avoid exceeding
> max. bit rate when muxing.

Do you have a sample that clearly shows failure if correct DTS handling
is ignored?

How does it work when embedded in formats that don't have DTS, like
Matroska or eac3to .sup files?


More information about the ffmpeg-devel mailing list