[FFmpeg-devel] [PATCH v5 0/6] Properly decode ogg metadata in ogg/flac and ogg/opus chained bitstreams

[Romain Beauxis]

Le lun. 17 févr. 2025 à 10:19, Romain Beauxis
<romain.beauxis at gmail.com> a écrit :
>
> This is a series of patches to allow proper decoding of ogg metadata in chained
> `ogg/flac` and `ogg/opus` streams.

I forgot to add: this patch set is also available for review at:
https://code.ffmpeg.org/FFmpeg/FFmpeg/pulls/6

> ## Changes since last version:
> - Opus processing moved back to `opus_header` to make sure that opus
>   demuxing is not impacted by those changes.
> - Generic decoding method implemented directly in `libavcodec/decode.c` to avoid
>   having each decoded handle the metadata injection.
>
> ### Summary of code changes:
>
> The changes in this patch series allow proper decoding of metadata associated
> with subsequent streams in chained ogg/flac and ogg/opus bitstream.
>
> This is done by intercepting ogg packets with comments in the ogg demuxer,
> parsing the comment block and attaching it as packed
> `AV_PKT_DATA_METADATA_UPDATE` side-data.
>
> The new metadata can then be unpacked in `libavcodec/decode.c` and attached to
> the next decoded frame.
>
> It is worth noting that is using a mechanism specific to ogg stream that seemed
> to only have been used for vorbis streams so far.
>
> Along with the changes are new FATE tests validating the implementation.
>
> ### Discussion and context:
>
> `ogg/opus` is a pretty popular combination of codec and encapsulation. In
> particular, it is widely used in Icecast streams, where chained streams are the
> norm because of the ogg specs requirement for inserting in-band metadata.
>
> `ogg/flac` streams are pretty important because there are perhaps the only
> combination of lossless audio codec and open-source container that allows for
> proper transmittion of lossless audio data accross systems such as Icecast,
> browser media tags and more.
>
> In the context of long-running audio streams, the ogg bitstream specs[1] have
> historically been very badly implemented. For each new track and each new
> metadata, the specs require the logical bitstream to come to a full `EOF` and
> then start with a full new logical stream.
>
> These specs have often been confused with a gobal EOF by most implementations.
>
> Furtunately, FFmpeg is a little better at that in that it is able to parse
> chained logical ogg bitstreams and properly output either encoded ogg packets or
> decoded audio.
>
> Current limitations with chained ogg streams in FFmpeg include:
> 1. Metadata from secondary chained `ogg/flac` and `ogg/opus` bitstreams are
>    ignored by the demuxer/decoder.
> 2. Secondary chained streams packet headers are silently removed by the
>    demuxer when decoding ogg/opus streams.
> 3. No adjustment underlying PTS or DTS: PTS and DTS of secondary chained
>    streams reset from their own logical stream initia value causing timestamp
>    discontinuity.
> 4. Chained bitstreams with more than one underlying type of content
>    (audio+video, etc) is not yet supported though this is a much less needed
>    feature.
>
> The changes in this patch series address issue #1.
>
> Future work could address the remaining issues.
> -- Romain
>
> [1]: https://xiph.org/ogg/doc/framing.html
>
> Romain Beauxis (6):
>   libavcodec/decode.c: intercept `AV_PKT_DATA_METADATA_UPDATE` packet
>     extra data, attach them to the next decoded frame.
>   tests: Add stream dump test API util.
>   libavformat/oggparseflac.c: Parse ogg/flac comments in new ogg
>     packets, add them to ogg stream new_metadata.
>   tests: Add chained ogg/flac stream dump test.
>   libavformat/oggparseopus.c: Parse comments from secondary chained
>     streams header packet.
>   tests: Add chained ogg/opus stream dump test.
>
>  libavcodec/decode.c                   |  20 +++
>  libavformat/oggparseflac.c            |  28 +++++
>  libavformat/oggparseopus.c            |  21 +++-
>  tests/Makefile                        |   3 +
>  tests/api/Makefile                    |   2 +-
>  tests/api/api-dump-stream-meta-test.c | 169 ++++++++++++++++++++++++++
>  tests/fate/ogg-flac.mak               |  11 ++
>  tests/fate/ogg-opus.mak               |  11 ++
>  8 files changed, 263 insertions(+), 2 deletions(-)
>  create mode 100644 tests/api/api-dump-stream-meta-test.c
>  create mode 100644 tests/fate/ogg-flac.mak
>  create mode 100644 tests/fate/ogg-opus.mak
>
> --
> 2.39.5 (Apple Git-154)
>