[FFmpeg-devel] [PATCH][RFC] avcodec/avutil: Add timeline side data

Derek Buitenhuis derek.buitenhuis at gmail.com
Sun Apr 1 17:39:07 EEST 2018

On 4/1/2018 12:44 AM, Michael Niedermayer wrote:
>> Not sure I follow what this has to do with timelines? There is no format that
>> exists that store timeline data interleaved, as far as I know - it is a
>> purely theoretical scenario.
> that surprises me. But if this case never occurs (or will occur) thats a good
> thing. Fewer cases to consider.

At least as far as I'm aware, yes.

>> What do you have in mind with regards to 'sharing'? I do not particularly
>> agree that chapters and edits should share stuff because they both happen to have
>> start and end times. They're quite different things, even if some formats have
>> conflated them (design flaw IMO).
> I did not think deeply about this but the obvious 3 options are to either
> give each its own struct and align the APIs used to access them
> to show this just with one function:
> AVTimelineList         *av_timeline_list_alloc(size_t timeline_count);
> AVChapterList          *av_chapterlist_alloc(size_t chapter_count);
> AVTimelineMetadataList *av_timeline_metadata_list_alloc(size_t metadata_count);
> AVTimelineCodecList    *av_timeline_codec_list_alloc(size_t codec_count);
> or to have some more common struct maybe similar to side data. It is after all
> some kind of side/meta data that is specific to timespans.
> or to put all in AVFormatContext as normal allocated arrays like chapters are
> currently.
> What i find a bit ugly is that each API uses the style popular at the time
> at which it is added and they are so far each completetly different even though
> they are semantically handling the same "unit" of data. that is information
> about "timespans".
> We have AVChapters which are a public field in AVFormatContext
> There is AV_PKT_DATA_STRINGS_METADATA which is a binary specified side data and
> there is the newly proposed timeline which is opaque sidedata with accessor functions
> Its not the inherent differences of the data that i want to push into a square hole
> but all the common parts that could be the same and are very different.
> If 2 things dont fit together iam certainly not intending to suggest to put them
> together. Nor that 2 things that have reason to be behind differnt APIs should be
> not.
> I mainly wanted to point out that our APIs for these things are quite
> different and that this should be thought about when adding a new API.
> Its not a specific cleanup or change i have in mind, more so i wanted to
> call attention to the inconsistency that there is between APIs
> so it is considered when adding a new API.
> If you determine that theres nothing that should be changed from what you
> intended previously, thats perfectly fine with me.

I agree that the APIs are annoyingly different, but that they should remain
separate APIs. The suggestion of aligned (same names, args, etc.) seems the
most reasonable to me - consistency.

I don't really think they can share some common structs, but they can certainly
be aligned.

However, some of the stuff you listed doesn't make sense to me:

* Codec changes are not part of timeline data; they're entirely separate things
  in every format I'm aware of. In MPEG-TS it just changes, in MOV/MP4 it is done
  via multiple 'stsd' atoms/boxes, which don't really carry any time related
  data - there is another atom/box that maps samples to a particular 'stsd',
  and that's it.
* What is AVTimelineMetadataList meant to be?

Lastly, what is the intent here? Is it to align all of these before timeline
support, or to deprecate and align after? Right now the API is align with
other side data APIs. I need to know which scenario this is before I go
ahead and implement this.

- Derek

More information about the ffmpeg-devel mailing list