[FFmpeg-devel] [PATCH] Matroska demuxer adds WebVTT support

[Clément Bœsch]

On Sat, Aug 10, 2013 at 12:37:03PM +0200, Nicolas George wrote:
> Le primidi 21 thermidor, an CCXXI, Clement Boesch a écrit :
> > If you're suggesting to use the WebM representation as the standard, we
> > have to deal with:
> > 
> >  - retro compat: changing the format of the packets for WebVTT format
> >    create version clashes between lavc & lavf (+ deprecation of the side
> >    data types)
> 
> That is always a burden, but I do not think it should come in the way of
> deciding a better design: if the design is bad, it will need to be changed
> sooner or later, and it is less work if it is done sooner.
> 
> In that particular case, I suspect we could just change the format of the
> packets and get away with it, although some additional safeguards would
> probably be better.
> 

We have a release with that packet form already.

> >  - move the complexity into lavf for webvtt muxing and demuxing
> 
> I suppose you mean "into lavc"? Otherwise I do not understand your sentence.
> Then, yes: better the complexity once in lavc than twice in lavf!
> 

My point is that the complexity will be twice in lavf: WebVTT demuxer will
have to craft a AVPacket in the WebM layout (not really more complex in
comparison to the current code I admit, even though I don't like doing
that), and the lavf muxer will have to parse strings from the AVPacket
data.

And on a side note, you'll also need to add some code in webvtt decoder to
skip those irrelevant information (which are currently accessible anyway
through the side data if you really want them).

> >  - create more hacks if webvtt is muxed differently in another format
> 
> We will deal with it if it ever happens. Hopefully, people who decide on a
> format will chose to adhere to the choices made for WebM, for
> compatibility's sake.
> 
> >    (will you oppose to a sane muxing of webvtt in NUT?)
> 
> You imply that WebM's format is insane. I do not agree at all. It is not
> perfect¹, but it rather good.
> 

> (1: it would have been better with 0 as a separator instead of LF and tags
> in front of each fields.)
> 

That was my point. After all the muxing operation only defines how to
merge those fields, and that's the ugly thing here. But sure, that's not
that much a problem.

> > OTOH, adding a special case only in WebM demuxer where a bad muxing
> > decision was made doesn't look broken. That way of muxing into WebM is
> > merely the result of two successive bad decisions, it doesn't look correct
> > to rely on its layout
> 
> The broken thing is the heavy use of side data. Side data is a hack to code
> whatever information is present somewhere but does not fit the common data
> structure. How do you mux side data into NUT? Into MP4? Into elementary
> streams? Answer: you do not, it does not exist.

Sure, and in MKV either. The main interest of having chapter & settings
separate into side data was purposely to avoid each muxers from parsing
them from the AVPacket to merge them in their desired way.

>                                                 Side data exist because one
> particular codec or container has a very specific feature to store one field
> that does not exist elsewhere and it was not deemed important enough to
> warrant a new field in AVPacket or AVFrame.
> 
> It does not apply there.
> 

I don't really consider the use of side data an abuse here, but simply a
different usage of it.

> > (and relying on the original WebVTT layout with timestamps will lead to
> > the ASS insanity you know: writing timestamps into demuxer etc).
> 
> No one is suggesting that, naturally.
> 

'Just wanted to be sure :)

> > Also note that the hack I wanted to remove with ASS/SSA was the insane
> > string *constructions* of SSA packets within the matroska demuxer, which
> > is not the case here: the text is extracted, and extra data put into side
> > data.
> 
> Granted, that was not exactly what I was referring to.
> 
> > Now if you insist on comparing with ASS/SSA, it might be a better solution
> > for that case to abstract the differences between ASS and SSA into side
> > data (like the ReadOrder),
> 
> That is exactly what I was referring to: ASS-in-Matroska and WebVTT-in-WebM
> share the exact same structure: technical_field, delimiter,
> technical-field... delimiter, actual_text.
> 

> And NO! it is not the demuxer's job to split the fields in the packet; a
> packet is a blob of arbitrary data, it should be totally opaque to the
> demuxer.
> 

WebVTT demuxer needs to split them anyway. So the only additional burden
code will remain in webm demuxer. Not doing this means having that parsing
burden in a lot of other places (see top of my reply).

> The job of splitting the fields in the packet is for the decoder. And the
> result should go, in the end, in specific fields in the new AVSubtitle2
> structure, not in the magical bag of holding called side data.
> 

So you want the guy proposing a relatively trivial demuxing of WebVTT in
WebM to wait (or implement) AVSubtitle2 and deal with all your concerns
about the current state?

That doesn't sound very wise to me, unless you can propose a patchset in
the following week... And I'm not even sure your changes proposition is
really an improvement over the current situation.

-- 
Clément B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130810/50ddcaeb/attachment.asc>