[FFmpeg-devel] [PATCH] Add flag to drop non-forced subtitle frames
Clément Bœsch
u at pkh.me
Sun May 25 11:18:57 CEST 2014
On Thu, May 22, 2014 at 07:39:11PM +0200, Oliver Fromme wrote:
> Hello,
>
> As I mentioned earlier this week, I'm trying to improve support
> for forced and partially forced subtitles. The patch below is
> the next step.
>
> The patch changes four files. I'll explain in detail what the
> changes are good for:
>
> libavcodec/avcodec.h -- Introduce a new codec flag for subtitle
> encoders, it's called CODEC_FLAG_FORCED_SUBS. When set, this
> flag indicates that the encoder should write only such subtitle
> frames that are marked as forced. Frames that are *not* forced
> will be ignored by the encoder.
>
> By the way, is there a rule for allocating bit masks for new
> flags? I just took the first unused one, that is 0x0080.
>
Yes, we generally introduce a gap between the last added one and the new
one, unless the last one added was FFmpeg specific in order to keep
compatibility with the fork Libav.
> libavcodec/options_table.h -- Add an option so the new flag can
> be set on the command line like this: -flags +forced_subs
>
> Before I continue, I need to explain some background: A single
> subtitle frame can consist of multiple rectangles, each one has
> its own forced flag. PGS (i.e. the Blu-Ray subtitle format)
> supports multiple rectangles, but VOBSUB (a.k.a. dvdsub) does
> not. So, if we have a subtitle frame with multiple rectangles,
> we have to merge them into one rectangle when encoding to dvdsub.
> The dvdsubenc encoder already does that, but it doesn't respect
> the forced flag yet. Now comes my patch ...
>
> libavcodec/dvdsubenc.c -- If CODEC_FLAG_FORCED_SUBS is set and
> there are multiple rectangles, we need to figure out which ones
> are forced and which ones are not, and then merge only the ones
> that are forced. The new variable "first" contains the index
> of the first rectangle that needs to be merged (otherwise it
> contains 0, so the behaviour is unchanged if the codec flag is
> not set).
>
> In all the following loops that iterate over the rectangles,
> any rectangles that are non-forced are skipped when the codec
> flag is set. That's all.
>
> Unfortunately, there's a special case that cannot be handled
> here in the encoder: When *all* rectangles of this subtitle
> are non-forced (and the codec flag is set). In this case we
> would have to skip all rectangles and produce an empty frame.
> This is not possible in the encoder ... The API expects that
> the codec's encode() function produces a frame for every input
> frame. Therefore, this case has to be handled at a higher
> level:
That sounds like something we will want in the next subtitles encoding
API… What happen if you actually do it?
>
> ffmpeg.c -- In the do_subtitle_out() function, we check if
> CODEC_FLAG_FORCED_SUBS is set. If it is, we check if *all*
> of the rectangles of the current subtitle frame are non-forced.
> If that's the case, we return right away, not calling the
> encoder at all, not calling write_frame(), not increasing the
> frame counter etc.
>
> You can test the new functionality with the file that I had
> uploaded on May 18th, called subt-forced-test.mkv. It
> contains one dvdsub subtitle track that has 991 frames, 2 of
> which are forced.
> For example, the following command could be used to "split"
> the subtitles in two output files: The first one contains
> all of them, the second one contains only the forced frames:
>
> $ ffmpeg -i subt-forced-test.mkv \
> -codec:s dvdsub test1.mkv \
> -codec:s dvdsub -flags:s +forced_subs test2.mkv
>
> $ mkvextract tracks test1.mkv 0:test1.idx
> $ mkvextract tracks test2.mkv 0:test2.idx
> $ grep -c timestamp test[12].idx
> test1.idx:991
> test2.idx:2
Just curious, what is this flag used for generally? Some forced
multi-language text annotation for the movie that probably anyone wants to
see?
>
> Voila.
>
> Best regards
> Oliver
>
>
> --- ffmpeg/libavcodec/avcodec.h.orig 2014-05-13 19:20:05.000000000 +0200
> +++ ffmpeg/libavcodec/avcodec.h 2014-05-21 18:52:28.000000000 +0200
Ideally if you could do a git commit and git format-patch -1 and send the
result that would be great. It will include your authorship, a commit
message and a description, so that's way simpler for us.
> @@ -758,6 +758,7 @@
> */
> #define CODEC_FLAG_MV0 0x0040
> #endif
> +#define CODEC_FLAG_FORCED_SUBS 0x0080 ///< Encode forced subtitles only.
> #if FF_API_INPUT_PRESERVED
> /**
> * @deprecated passing reference-counted frames to the encoders replaces this
> --- ffmpeg/libavcodec/options_table.h.orig 2014-05-13 19:20:05.000000000 +0200
> +++ ffmpeg/libavcodec/options_table.h 2014-05-21 18:55:15.000000000 +0200
> @@ -60,6 +60,7 @@
> #if FF_API_MV0
> {"mv0", "always try a mb with mv=<0,0>", 0, AV_OPT_TYPE_CONST, {.i64 = CODEC_FLAG_MV0 }, INT_MIN, INT_MAX, V|E, "flags"},
> #endif
> +{"forced_subs", "encode forced subtitle captions only", 0, AV_OPT_TYPE_CONST, {.i64 = CODEC_FLAG_FORCED_SUBS }, INT_MIN, INT_MAX, S|E, "flags"},
If you could update doc/codecs.texi as well that would be great
> #if FF_API_INPUT_PRESERVED
> {"input_preserved", NULL, 0, AV_OPT_TYPE_CONST, {.i64 = CODEC_FLAG_INPUT_PRESERVED }, INT_MIN, INT_MAX, 0, "flags"},
> #endif
> --- ffmpeg/libavcodec/dvdsubenc.c.orig 2014-05-16 23:35:29.000000000 +0200
> +++ ffmpeg/libavcodec/dvdsubenc.c 2014-05-21 20:03:55.000000000 +0200
> @@ -260,6 +260,8 @@
> uint8_t *vrect_data = NULL;
> int x2, y2;
> int forced = 0;
> + int first = 0;
> + int want_forced_subs = (avctx->flags & CODEC_FLAG_FORCED_SUBS) != 0;
>
> if (rects == 0 || h->rects == NULL)
> return AVERROR(EINVAL);
> @@ -271,9 +273,12 @@
> /* Mark this subtitle forced if any of the rectangles is forced. */
> for (i = 0; i < rects; i++)
> if ((h->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED) != 0) {
> + first = i;
> forced = 1;
> break;
> }
> + if (!want_forced_subs)
> + first = 0;
> vrect = *h->rects[0];
>
> if (rects > 1) {
> @@ -281,9 +286,11 @@
> rectangle containing all actual rectangles.
> The data of the rectangles will be copied later, when the palette
> is decided, because the rectangles may have different palettes. */
> - int xmin = h->rects[0]->x, xmax = xmin + h->rects[0]->w;
> - int ymin = h->rects[0]->y, ymax = ymin + h->rects[0]->h;
> + int xmin = h->rects[first]->x, xmax = xmin + h->rects[first]->w;
> + int ymin = h->rects[first]->y, ymax = ymin + h->rects[first]->h;
> for (i = 1; i < rects; i++) {
> + if (want_forced_subs && !(h->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED))
> + continue;
> xmin = FFMIN(xmin, h->rects[i]->x);
> ymin = FFMIN(ymin, h->rects[i]->y);
> xmax = FFMAX(xmax, h->rects[i]->x + h->rects[i]->w);
> @@ -298,12 +305,18 @@
>
> /* Count pixels outside the virtual rectangle as transparent */
> global_palette_hits[0] = vrect.w * vrect.h;
> - for (i = 0; i < rects; i++)
> + for (i = first; i < rects; i++) {
> + if (want_forced_subs && !(h->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED))
> + continue;
> global_palette_hits[0] -= h->rects[i]->w * h->rects[i]->h;
> + }
> }
>
> - for (i = 0; i < rects; i++)
> + for (i = first; i < rects; i++) {
> + if (want_forced_subs && !(h->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED))
> + continue;
> count_colors(avctx, global_palette_hits, h->rects[i]);
> + }
> select_palette(avctx, out_palette, out_alpha, global_palette_hits);
>
> if (rects > 1) {
> @@ -311,7 +324,9 @@
> return AVERROR(ENOMEM);
> vrect.pict.data [0] = vrect_data;
> vrect.pict.linesize[0] = vrect.w;
> - for (i = 0; i < rects; i++) {
> + for (i = first; i < rects; i++) {
> + if (want_forced_subs && !(h->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED))
> + continue;
> build_color_map(avctx, cmap, (uint32_t *)h->rects[i]->pict.data[1],
> out_palette, out_alpha);
> copy_rectangle(&vrect, h->rects[i], cmap);
> --- ffmpeg/ffmpeg.c.orig 2014-05-08 19:20:05.000000000 +0200
> +++ ffmpeg/ffmpeg.c 2014-05-21 19:05:33.000000000 +0200
> @@ -769,6 +769,7 @@
> AVCodecContext *enc;
> AVPacket pkt;
> int64_t pts;
> + int forced = 0;
>
> if (sub->pts == AV_NOPTS_VALUE) {
> av_log(NULL, AV_LOG_ERROR, "Subtitle packets must have a pts\n");
> @@ -779,6 +780,16 @@
>
> enc = ost->st->codec;
>
> + if (enc->flags & CODEC_FLAG_FORCED_SUBS) {
> + for (i = 0; i < sub->num_rects; i++)
> + if (sub->rects[i]->flags & AV_SUBTITLE_FLAG_FORCED) {
> + forced = 1;
> + break;
> + }
> + if (!forced)
> + return;
> + }
> +
> if (!subtitle_out) {
> subtitle_out = av_malloc(subtitle_out_max_size);
> }
The code looks fine to me, but I haven't tested yet and I'm not the
maintainer of that file.
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
--
Clément B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140525/9988cccb/attachment.asc>
More information about the ffmpeg-devel
mailing list