[FFmpeg-devel] [PATCH] new generic metadata API

Michael Niedermayer michaelni
Sun Sep 28 00:27:40 CEST 2008


On Sat, Sep 27, 2008 at 04:03:53PM +0200, Aurelien Jacobs wrote:
> On Tue, 23 Sep 2008 15:21:43 +0200
> Michael Niedermayer <michaelni at gmx.at> wrote:
[...]

> > [note, also see the nut-devel ML for some related change suggestion i just
> >  posted]
> 
> Seen it.
> I've added generic flattening and un-flattening support, following your
> proposed nut scheme. The exact formatting could be changed latter if
> a different scheme is accepted.

What does it do with 2
Author tags? does it convert to Author / Author2?
Similarly, does it remove the '2' postfix when unflattening?
does it prefix non standard parts with 'X-' when muxing nut?

One problem there will be if it does not handle postfixes is

Author "Michael"
    EMAIL   "foo"
Author "Aurel"
    EMAIL   "bar"

Author  Michael
Author/EMAIL foo
Author  Aurel
Author/EMAIL bar

the result is sensitive to the order of elements


[...]
> > >  static int write_globalinfo(NUTContext *nut, ByteIOContext *bc){
> > >      AVFormatContext *s= nut->avf;
> > >      ByteIOContext *dyn_bc;
> > > @@ -454,9 +461,10 @@
> > >      if(ret < 0)
> > >          return ret;
> > >  
> > > -    if(s->title    [0]) count+= add_info(dyn_bc, "Title"    , s->title);
> > > -    if(s->author   [0]) count+= add_info(dyn_bc, "Author"   , s->author);
> > > -    if(s->copyright[0]) count+= add_info(dyn_bc, "Copyright", s->copyright);
> > > +    count += add_tag(dyn_bc, &s->metadata, "Title");
> > > +    count += add_tag(dyn_bc, &s->metadata, "Author");
> > > +    count += add_tag(dyn_bc, &s->metadata, "Copyright");
> > > +    count += add_tag(dyn_bc, &s->metadata, "Description");
> > >      if(!(s->streams[0]->codec->flags & CODEC_FLAG_BITEXACT))
> > >                          count+= add_info(dyn_bc, "Encoder"  , LIBAVFORMAT_IDENT);
> > >  
> > 
> > well, thats not supporting generic metadata, it just hardcodes 4 cases
> 
> My goal was mainly to add the new API, and convert all (de)muxers
> so that they don't use the old API anymore. I didn't try to implement
> support for every possible kind of metadata feature in every (de)muxers.
> Some (de)muxers could be further improved, but that's out of the scope of
> this patch (matroska muxer don't even support muxing tags at all !).
> Anyway, I've improved nut muxing, so it now really supports generic
> metadata, with nested tags, and laguage code.
> 
> Attached patch is not split like in my original post (to ease testing),
> but it is still intended to be applied in several steps.
> You can test it with the following samples:
>   http://samples.mplayerhq.hu/Matroska/nested_tags.mka
>   http://samples.mplayerhq.hu/Matroska/nested_tags_lang.mka
> Converting last one to nut (flattening tags) and then ffplay -stats
> on the nut file (un-flattening) will show this:
> 

> BAND: Metallica
>   LEAD_PERFORMER (ger): Lars Ulrich
>     URL: http://www.metallica.com/Band/lars.asp
>     ADDRESS: Gentofte, Denmark
>       DATE: 1963-12-06
>       DATEEND: 1970-12-06
>     ADDRESS (eng): Whereever, US
>       DATE: 1970-12-07
>     INSTRUMENT: Drums
> Encoder: Lavf52.23.0
> 
> (the lang values where added by me and are not supposed to be
> meaningful, there are just here for testing purpose)

ok, but i mention it anyway because above is semantically wrong
a address in denmark written in english should have a language=english
if you write the address in chinese it would be langugae=chinese, the address
still would refer to denmark though.


[...]
> Index: libavformat/avformat.h
> ===================================================================
> --- libavformat/avformat.h	(revision 15392)
> +++ libavformat/avformat.h	(working copy)
> @@ -22,8 +22,8 @@
>  #define AVFORMAT_AVFORMAT_H
>  
>  #define LIBAVFORMAT_VERSION_MAJOR 52
> -#define LIBAVFORMAT_VERSION_MINOR 22
> -#define LIBAVFORMAT_VERSION_MICRO  1
> +#define LIBAVFORMAT_VERSION_MINOR 23
> +#define LIBAVFORMAT_VERSION_MICRO  0
>  
>  #define LIBAVFORMAT_VERSION_INT AV_VERSION_INT(LIBAVFORMAT_VERSION_MAJOR, \
>                                                 LIBAVFORMAT_VERSION_MINOR, \

> @@ -314,6 +314,20 @@
>      struct AVInputFormat *next;
>  } AVInputFormat;
>  
> +typedef struct AVMetadataTag AVMetadataTag;
> +
> +typedef struct AVMetadata {
> +    unsigned int nb;
> +    AVMetadataTag **list;
> +} AVMetadata;
> +
> +struct AVMetadataTag {
> +    char *name;
> +    char *value;
> +    char *lang;
> +    AVMetadata nested;
> +};
> +
>  enum AVStreamParseType {
>      AVSTREAM_PARSE_NONE,
>      AVSTREAM_PARSE_FULL,       /**< full parsing and repack */

Is it needed to have these structs in a public header?
Making the structs less public would allow more changes without breaking
the API/ABI, we for example might one day want to use AVTree for it, and or
might even want to switch to a flat internal format.
Also, maybe the double indirection in AVMetadata.list could be avoided if
it where non public.
But maybe iam missing some reason why this wouldnt be easily possible ...


> @@ -395,7 +409,9 @@
>       */
>      int64_t duration;
>  
> +#if LIBAVFORMAT_VERSION_MAJOR < 53
>      char language[4]; /** ISO 639 3-letter language code (empty string if undefined) */
> +#endif
>  
>      /* av_read_frame() support */
>      enum AVStreamParseType need_parsing;
> @@ -414,9 +430,9 @@
>  
>  #if LIBAVFORMAT_VERSION_INT < (53<<16)
>      int64_t unused[4+1];
> -#endif
>  
>      char *filename; /**< source filename of the stream */
> +#endif
>  
>      int disposition; /**< AV_DISPOSITION_* bitfield */
>  

ok


[...]
> @@ -457,7 +478,10 @@
>      int id;                 ///< Unique id to identify the chapter
>      AVRational time_base;   ///< Timebase in which the start/end timestamps are specified
>      int64_t start, end;     ///< chapter start/end time in time_base units
> +#if LIBAVFORMAT_VERSION_MAJOR < 53
>      char *title;            ///< chapter title
> +#endif
> +    AVMetadata metadata;
>  } AVChapter;
>  
>  #define MAX_STREAMS 20
> @@ -481,6 +505,7 @@
>      char filename[1024]; /**< input or output filename */
>      /* stream info */
>      int64_t timestamp;
> +#if LIBAVFORMAT_VERSION_MAJOR < 53
>      char title[512];
>      char author[512];
>      char copyright[512];
> @@ -489,6 +514,7 @@
>      int year;  /**< ID3 year, 0 if none */
>      int track; /**< track number, 0 if none */
>      char genre[32]; /**< ID3 genre */
> +#endif
>  
>      int ctx_flags; /**< format specific flags, see AVFMTCTX_xx */
>      /* private data for pts handling (do not modify directly) */

ok


[..]
> +/**
> + * Count the number of tags present in a metadata set among a list of names.
> + * @param metadata metadata set
> + * @param names tag name list (NULL terminated)
> + * @return the count of available tags
> + */
> +int av_metadata_has_tags(AVMetadata *metadata, const char **names);
> +

i would suggest "count" instead of "has"


> +/**
> + * Copy the content of a full metadata set.
> + * @param out destination of the copy
> + * @param in source of the copy
> + * @return 0 if OK. AVERROR_xxx if error.
> + */
> +int av_metadata_copy(AVMetadata *out, AVMetadata *in);

clone is maybe a better term than copy


[...]
> @@ -2407,6 +2406,311 @@
>      return chapter;
>  }
>  
> +#if LIBAVFORMAT_VERSION_MAJOR < 53
[...]
> +#endif
> +
> +static const char const *metadata_mapping[][6] = {
> +    { "author",   "artist", "creator", "written_by", "lead_performer" },
> +    { "comment",  "description" },
> +    { "album",    "albumtitle" },
> +    { "year",     "date_written", "date_released" },
> +    { "track",    "tracknumber", "part_number" },
> +};
> +
> +static AVMetadataTag *find_metadata_tag(AVMetadata *metadata,
> +                                        const char *name, const char *lang,
> +                                        int strict_lang, int name_len)
> +{
> +    int i;
> +    for (i=0; i<metadata->nb; i++)
> +        if (!strncasecmp(metadata->list[i]->name, name, name_len))
> +            if ((!lang && (!strict_lang || !metadata->list[i]->lang)) ||
> +                (lang && metadata->list[i]->lang &&
> +                 !strncasecmp(metadata->list[i]->lang, lang, 3)))
> +                return metadata->list[i];
> +    return NULL;
> +}
> +
> +static const char *extract_metadata_lang(const char *lang,
> +                                         const char *name, int *len)
> +{
> +    if (!lang && *len>4 && name[*len-4] == '-' && islower(name[*len-3])
> +        && islower(name[*len-2]) && islower(name[*len-1])) {
> +        *len -= 4;
> +        return name + *len + 1;
> +    }
> +    return lang;
> +}
> +
> +AVMetadataTag *av_metadata_set_tag(AVMetadata *metadata, int flags,
> +                                   const char *name, const char *value,
> +                                   const char *lang)
> +{
> +    AVMetadataTag *tag = NULL;
> +    const char *ptr, *lang2;
> +    int len;
> +    if (!name || !value || !*name || !*value)
> +        return NULL;
> +    if (flags & AV_METADATA_FLAG_AUTONEST && (ptr = strchr(name, '/'))) {
> +        len = ptr - name;
> +        lang2 = extract_metadata_lang(lang, name, &len);
> +        tag = find_metadata_tag(metadata, name, lang2, 1, len);
> +        if (tag)
> +            return av_metadata_set_tag(&tag->nested, flags, ptr+1, value, lang);
> +        return NULL;
> +    }
> +    len = strlen(name);
> +    lang = extract_metadata_lang(lang, name, &len);
> +    if (!(flags & AV_METADATA_FLAG_DUPLICATE))
> +        tag = find_metadata_tag(metadata, name, lang, 1, len);
> +    if (!tag) {
> +        tag = av_mallocz(sizeof(*tag));
> +        if (!tag)
> +            return NULL;
> +        dynarray_add(&metadata->list, &metadata->nb, tag);
> +    } else {
> +        av_free(tag->name);
> +        av_free(tag->value);
> +        av_free(tag->lang);
> +    }
> +    tag->name  = av_strdup(name);
> +    tag->value = av_strdup(value);
> +    tag->lang  = av_strdup(lang);
> +    tag->name[len] = 0;
> +    return tag;
> +}
> +
> +AVMetadataTag *ff_metadata_set_tag(AVFormatContext *s, AVMetadata *metadata,
> +                                   int flags, const char *name,
> +                                   const char *value, const char *lang)
> +{
> +    /* after major version bump, this internal wrapper function
> +       should be removed. */
> +#if LIBAVFORMAT_VERSION_MAJOR < 53
[...]
> +#endif
> +
> +    return av_metadata_set_tag(metadata, flags, name, value, lang);
> +}
> +
> +char *av_metadata_get_tag(AVMetadata *metadata, const char *name, const char *lang)
> +{
> +    AVMetadataTag *tag = find_metadata_tag(metadata, name, lang, 0, INT_MAX);
> +    int i, j;
> +
> +    for (i=0; !tag && i<sizeof(metadata_mapping)/sizeof(*metadata_mapping); i+=2)
> +        if (!strcasecmp(metadata_mapping[i][0], name))
> +            for (j=1; !tag && metadata_mapping[i][j]; j++)
> +                tag = find_metadata_tag(metadata, metadata_mapping[i][j], lang, 0, INT_MAX);
> +
> +    return tag ? tag->value : NULL;
> +}
> +
> +int av_metadata_has_tags(AVMetadata *metadata, const char **names)
> +{
> +    const char **name;
> +    int count = 0;
> +    for (name=names; *name; name++)
> +        count += !!av_metadata_get_tag(metadata, *name, NULL);
> +    return count;
> +}
> +
> +int av_metadata_copy(AVMetadata *out, AVMetadata *in)
> +{
> +    int i;
> +    for (i=0; i<in->nb; i++) {
> +        AVMetadataTag *tag;
> +        tag = av_metadata_set_tag(out, AV_METADATA_FLAG_DUPLICATE,
> +                                  in->list[i]->name, in->list[i]->value,
> +                                  in->list[i]->lang);
> +        if (!tag || av_metadata_copy(&tag->nested, &in->list[i]->nested))
> +            return AVERROR(ENOMEM);
> +    }
> +    return 0;
> +}
> +
> +int av_metadata_flatten(AVMetadata *out, AVMetadata *in, char *prefix)
> +{
> +    AVMetadataTag *tag;
> +    char buffer[1024];
> +    int i, num, pos;
> +    for (i=0; i<in->nb; i++) {
> +        if (strchr(in->list[i]->name, '/') || strchr(in->list[i]->name, '-')) {
> +            av_log(NULL, AV_LOG_ERROR, "Invalid tag name.\n");
> +            continue;
> +        }
> +        num = 1;
> +        *buffer = 0;
> +        if (prefix)
> +            snprintf(buffer, sizeof(buffer), "%s/", prefix);
> +        av_strlcat(buffer, in->list[i]->name, sizeof(buffer));
> +        pos = strlen(buffer);
> +        if (in->list[i]->lang)
> +            snprintf(buffer+pos, sizeof(buffer)-pos, "-%s", in->list[i]->lang);
> +        while (find_metadata_tag(out, buffer, NULL, 0, INT_MAX)) {
> +            int n = snprintf(buffer+pos, sizeof(buffer)-pos, "%d", ++num);
> +            if (in->list[i]->lang)
> +                snprintf(buffer+pos+n, sizeof(buffer)-pos-n,
> +                         "-%s", in->list[i]->lang);
> +        }
> +        tag = av_metadata_set_tag(out, 0, buffer,
> +                                  in->list[i]->value, in->list[i]->lang);
> +        if (!tag || av_metadata_flatten(out, &in->list[i]->nested, buffer))
> +            return AVERROR(ENOMEM);
> +    }
> +    return 0;
> +}
> +
> +void av_metadata_empty(AVMetadata *metadata)
> +{
> +    int i;
> +    for (i=0; i<metadata->nb; i++) {
> +        av_metadata_empty(&metadata->list[i]->nested);
> +        av_free(metadata->list[i]->name);
> +        av_free(metadata->list[i]->value);
> +        av_free(metadata->list[i]);
> +    }
> +    av_freep(&metadata->list);
> +    metadata->nb = 0;
> +}
> +
> +#if LIBAVFORMAT_VERSION_MAJOR < 53
[...]
> +#endif

maybe all that stuff could be put in a metadata.c instead of utils.c

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Complexity theory is the science of finding the exact solution to an
approximation. Benchmarking OTOH is finding an approximation of the exact
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20080928/bdc2970d/attachment.pgp>



More information about the ffmpeg-devel mailing list