[FFmpeg-devel] [PATCH]Basic XSUB encoder (take 2)

Björn Axelsson gecko
Mon Feb 2 22:55:05 CET 2009


On Sun, 1 Feb 2009, Bj?rn Axelsson wrote:

> This is a cleanup of a patch submitted earlier by Steve Lhomme / DivX
> Corp [1].
>
> With this command line I can encode an avi with subtitles which displays
> somewhat correctly on a PS3 and in ffplay:
> ./ffmpeg -i Starship_Troopers.vob -s 1280x536 -vtag DX50 -vcodec mpeg4 -an
> -scodec xsub -y test.avi
>
> Explanations and remaining issues:
> * The -s parameter is for the PS3 which I can't get to play files
> with lower resolution when encoded with ffmpeg.
> * The PS3 places the subs on the middle of the screen, possibly because of
>  the scaling of the video stream.
> * ffplay places the subs on the left hand side of the middle of the
> screen.

Never mind the scaling issues. I misinterpreted an unrelated issue between
the PS3 and the media server. Scaling is not necessary.

There's still an issue with ffmpeg and rescaling of video with subtitles,
but it is unrelated to this patch.

> * ffplay seems to get the timing of the subtitles wrong. Since it displays
> the original vob subtitles correctly, and the PS3 displays the xsub
> subtitles correctly my guess is that there's something wrong with demuxing
> or decoding of xsubs in ffmpeg.

Also unrelated to this patch, I think.

> * An extra field for the absolute pts is needed in AVSubtitle for the
> encoder. Any ideas for a better solution are welcome.

No one mentioned this in the first review, but it still is an ugly hack. I
did clean it up and document it in this version, though.

> The sample is from:
> http://samples.mplayerhq.hu/MPEG-VOB/ClosedCaptions/Starship_Troopers.vob
>
> The patch passes make test but make checkheaders fails on alsa-audio.h
> which I doubt is my fault...

Still passes make test, but make checkheaders fails on vdpau.h with
current svn...

I believe all issues raised so far have been dealt with in this patch.

-- 
Bj?rn Axelsson
-------------- next part --------------
Index: ffmpeg.c
===================================================================
--- ffmpeg.c.orig	2009-02-02 22:38:27.000000000 +0100
+++ ffmpeg.c	2009-02-02 22:39:14.000000000 +0100
@@ -814,6 +814,7 @@
         nb = 1;
 
     for(i = 0; i < nb; i++) {
+        sub->pts = av_rescale_q(pts, ist->st->time_base, AV_TIME_BASE_Q);
         subtitle_out_size = avcodec_encode_subtitle(enc, subtitle_out,
                                                     subtitle_out_max_size, sub);
 
Index: libavcodec/Makefile
===================================================================
--- libavcodec/Makefile.orig	2009-02-02 22:38:27.000000000 +0100
+++ libavcodec/Makefile	2009-02-02 22:39:14.000000000 +0100
@@ -251,6 +251,7 @@
 OBJS-$(CONFIG_XAN_WC4_DECODER)         += xan.o
 OBJS-$(CONFIG_XL_DECODER)              += xl.o
 OBJS-$(CONFIG_XSUB_DECODER)            += xsubdec.o
+OBJS-$(CONFIG_XSUB_ENCODER)            += xsubenc.o
 OBJS-$(CONFIG_XVMC)                    += xvmcvideo.o
 OBJS-$(CONFIG_ZLIB_DECODER)            += lcldec.o
 OBJS-$(CONFIG_ZLIB_ENCODER)            += lclenc.o
Index: libavcodec/xsubenc.c
===================================================================
--- /dev/null	1970-01-01 00:00:00.000000000 +0000
+++ libavcodec/xsubenc.c	2009-02-02 22:39:14.000000000 +0100
@@ -0,0 +1,193 @@
+/*
+ * DivX (XSUB) subtitle encoder
+ * Copyright (c) 2005 DivX, Inc.
+ * Copyright (c) 2009 Bjorn Axelsson
+ *
+ * This file is part of FFmpeg.
+ *
+ * FFmpeg is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * FFmpeg is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with FFmpeg; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
+ */
+
+#include "avcodec.h"
+#include "bytestream.h"
+#include "bitstream.h"
+
+/* For unknown reasons we pad the subtitles with two pixels on either side */
+#define MARGIN 2
+
+static inline int getcolor(const uint8_t *row, int w, int x)
+{
+    if (x >= MARGIN && x < w+MARGIN)
+        return row[x-MARGIN] & 3;
+    else
+        return 0;
+}
+
+static void xsub_encode_rle(uint8_t **buffer, int bufsize,
+        const uint8_t *bitmap, int linesize, int w, int h)
+{
+    PutBitContext pb;
+    int x, y, len, x1, color;
+    /* Adds 2 pixels margin on either side + enforces width to multiple of 2 */
+    int w2 = ((w + 1) & ~1) + MARGIN * 2;
+
+    init_put_bits(&pb, *buffer, bufsize);
+
+    for (y = 0; y < h; y++) {
+        x = 0;
+        while (x < w2) {
+            x1 = x;
+            color = getcolor(bitmap, w, x1++);
+            while (x1 < w2 && getcolor(bitmap, w, x1) == color)
+                x1++;
+            len = x1 - x;
+
+            // Run can't be longer than 255, unless it is the rest of a row
+            if (x1 < w2)
+                len = FFMIN(len, 255);
+
+            if (len <= 255)
+                put_bits(&pb, 2 + ((ff_log2_tab[len] >> 1) << 2), len);
+            else
+                put_bits(&pb, 14, 0);
+            put_bits(&pb, 2, color);
+
+            x += len;
+        }
+
+        // byte align row
+        if (put_bits_count(&pb) & 4)
+            put_bits(&pb, 4, 0);
+
+        bitmap += linesize;
+    }
+
+    *buffer += put_bits_count(&pb) / 8;
+    flush_put_bits(&pb);
+}
+
+static const int tc_divs[3] = { 1000, 60, 60 };
+static int make_tc(uint64_t ms, int *tc)
+{
+    int i;
+    for (i=0; i<3; i++) {
+        tc[i] = ms % tc_divs[i];
+        ms /= tc_divs[i];
+    }
+    tc[3] = ms;
+    return ms > 99;
+}
+
+static int xsub_encode(AVCodecContext *avctx, unsigned char *buf,
+                       int bufsize, void *data)
+{
+    AVSubtitle *h = (AVSubtitle *)data;
+    uint64_t startTime = h->pts / 1000; // FIXME: need better solution...
+    uint64_t endTime = startTime + h->end_display_time - h->start_display_time;
+    int start_tc[4], end_tc[4];
+
+    uint8_t *hdr = (uint8_t *)buf;
+    uint8_t *rlelenptr;
+    uint8_t *q;
+
+    uint16_t width, height;
+
+    int i;
+
+    if (h->num_rects == 0 || h->rects == NULL)
+        return -1;
+
+    // TODO: support multiple rects
+    if (h->num_rects > 1)
+        av_log(avctx, AV_LOG_WARNING, "Only single rects supported (%d in subtitle.)\n", h->num_rects);
+
+    // TODO: render text-based subtitles
+    if (!h->rects[0]->pict.data[0] || !h->rects[0]->pict.data[1]) {
+        av_log(avctx, AV_LOG_WARNING, "No subtitle bitmap available.\n");
+        return -1;
+    }
+
+    // TODO: color reduction, similar to dvdsub encoder
+    if (h->rects[0]->nb_colors > 4)
+        av_log(avctx, AV_LOG_WARNING, "No more than 4 subtitle colors supported (%d found.)\n", h->rects[0]->nb_colors);
+
+    if(make_tc(startTime, start_tc) || make_tc(endTime, end_tc)) {
+        av_log(avctx, AV_LOG_WARNING, "Time code >= 100 hours.\n");
+        return -1;
+    }
+    snprintf(hdr, 28,
+        "[%02d:%02d:%02d.%03d-%02d:%02d:%02d.%03d]",
+        start_tc[3], start_tc[2], start_tc[1], start_tc[0],
+        end_tc[3],   end_tc[3],   end_tc[1],   end_tc[0]);
+    hdr += 27;
+
+    // Width and height must probably be multiple of 2.
+    // 2 pixels required on either side of subtitle.
+    // FIXME: Why?
+    width = ((h->rects[0]->w + 1) & ~1) + MARGIN * 2;
+    height = (h->rects[0]->h + 1) & ~1;
+
+    bytestream_put_le16(&hdr, width);
+    bytestream_put_le16(&hdr, height);
+    bytestream_put_le16(&hdr, h->rects[0]->x);
+    bytestream_put_le16(&hdr, h->rects[0]->y);
+    bytestream_put_le16(&hdr, h->rects[0]->x + width);
+    bytestream_put_le16(&hdr, h->rects[0]->y + height);
+
+    rlelenptr = hdr;
+    hdr+=2;
+
+    // Palette
+    for (i=0; i<4; i++)
+        bytestream_put_be24(&hdr, ((uint32_t *)h->rects[0]->pict.data[1])[i]);
+
+    // Bitmap
+    q = hdr;
+    xsub_encode_rle(&q, bufsize - (q-buf),
+            h->rects[0]->pict.data[0],
+            h->rects[0]->pict.linesize[0]*2,
+            h->rects[0]->w, (h->rects[0]->h + 1) / 2);
+    bytestream_put_le16(&rlelenptr, q-hdr); // Length of first field
+
+    xsub_encode_rle(&q, bufsize - (q-buf),
+            h->rects[0]->pict.data[0] + h->rects[0]->pict.linesize[0],
+            h->rects[0]->pict.linesize[0]*2,
+            h->rects[0]->w, h->rects[0]->h / 2);
+
+    // Enforce total height to be be multiple of 2
+    if (h->rects[0]->h & 1)
+        bytestream_put_le16(&q, 0); // RLE empty line
+
+    return q-buf;
+}
+
+static av_cold int xsub_encoder_init(AVCodecContext *avctx)
+{
+    if (!avctx->codec_tag)
+        avctx->codec_tag = MKTAG('D','X','S','B');
+
+    return 0;
+}
+
+AVCodec xsub_encoder = {
+    "xsub",
+    CODEC_TYPE_SUBTITLE,
+    CODEC_ID_XSUB,
+    0,
+    xsub_encoder_init,
+    xsub_encode,
+    NULL,
+    .long_name = NULL_IF_CONFIG_SMALL("DivX subtitles (XSUB)"),
+};
Index: libavcodec/allcodecs.c
===================================================================
--- libavcodec/allcodecs.c.orig	2009-02-02 22:38:38.000000000 +0100
+++ libavcodec/allcodecs.c	2009-02-02 22:39:50.000000000 +0100
@@ -282,7 +282,7 @@
     /* subtitles */
     REGISTER_ENCDEC  (DVBSUB, dvbsub);
     REGISTER_ENCDEC  (DVDSUB, dvdsub);
-    REGISTER_DECODER (XSUB, xsub);
+    REGISTER_ENCDEC  (XSUB, xsub);
 
     /* external libraries */
     REGISTER_ENCDEC  (LIBAMR_NB, libamr_nb);
Index: libavcodec/avcodec.h
===================================================================
--- libavcodec/avcodec.h.orig	2009-02-02 22:38:27.000000000 +0100
+++ libavcodec/avcodec.h	2009-02-02 22:39:14.000000000 +0100
@@ -30,7 +30,7 @@
 #include "libavutil/avutil.h"
 
 #define LIBAVCODEC_VERSION_MAJOR 52
-#define LIBAVCODEC_VERSION_MINOR 11
+#define LIBAVCODEC_VERSION_MINOR 12
 #define LIBAVCODEC_VERSION_MICRO  0
 
 #define LIBAVCODEC_VERSION_INT  AV_VERSION_INT(LIBAVCODEC_VERSION_MAJOR, \
@@ -2431,6 +2431,7 @@
     uint32_t end_display_time; /* relative to packet pts, in ms */
     unsigned num_rects;
     AVSubtitleRect **rects;
+    uint64_t pts;    ///< Same as packet pts, in AV_TIME_BASE
 } AVSubtitle;
 
 
Index: libavformat/avienc.c
===================================================================
--- libavformat/avienc.c.orig	2009-02-02 22:38:28.000000000 +0100
+++ libavformat/avienc.c	2009-02-02 22:39:14.000000000 +0100
@@ -82,6 +82,9 @@
     if (type == CODEC_TYPE_VIDEO) {
         tag[2] = 'd';
         tag[3] = 'c';
+    } else if (type == CODEC_TYPE_SUBTITLE) {
+        tag[2] = 's';
+        tag[3] = 'b';
     } else {
         tag[2] = 'w';
         tag[3] = 'b';
@@ -213,8 +216,10 @@
         case CODEC_TYPE_AUDIO: put_tag(pb, "auds"); break;
 //        case CODEC_TYPE_TEXT : put_tag(pb, "txts"); break;
         case CODEC_TYPE_DATA : put_tag(pb, "dats"); break;
+        case CODEC_TYPE_SUBTITLE: put_tag(pb, "vids"); break;
         }
-        if(stream->codec_type == CODEC_TYPE_VIDEO)
+        if(stream->codec_type == CODEC_TYPE_VIDEO
+                || stream->codec_type == CODEC_TYPE_SUBTITLE)
             put_le32(pb, stream->codec_tag);
         else
             put_le32(pb, 1);
@@ -254,6 +259,7 @@
         strf = start_tag(pb, "strf");
         switch(stream->codec_type) {
         case CODEC_TYPE_VIDEO:
+        case CODEC_TYPE_SUBTITLE:
             put_bmp_header(pb, stream, codec_bmp_tags, 0);
             break;
         case CODEC_TYPE_AUDIO:
Index: Changelog
===================================================================
--- Changelog.orig	2009-02-02 22:38:28.000000000 +0100
+++ Changelog	2009-02-02 22:39:14.000000000 +0100
@@ -146,6 +146,7 @@
 - hybrid WavPack support
 - R3D REDCODE demuxer
 - ALSA support for playback and record
+- DivX (XSUB) subtitle encoder
 
 version 0.4.9-pre1:
 
Index: doc/general.texi
===================================================================
--- doc/general.texi.orig	2009-02-02 22:38:39.000000000 +0100
+++ doc/general.texi	2009-02-02 22:41:11.000000000 +0100
@@ -455,7 +455,7 @@
 @item ASS/SSA      @tab X @tab X
 @item DVB          @tab X @tab X @tab X @tab X @tab X
 @item DVD          @tab X @tab X @tab X @tab X @tab X
- at item XSUB         @tab   @tab   @tab   @tab X @tab
+ at item XSUB         @tab X @tab X @tab X @tab X @tab X
 @end multitable
 
 @code{X} means that the feature is supported.



More information about the ffmpeg-devel mailing list