[FFmpeg-devel] [PATCH 0/7] RFC: complete rework of s337m support

[Nicolas Gaullier]

>De : ffmpeg-devel <ffmpeg-devel-bounces at ffmpeg.org> de la part de Lynne via ffmpeg-devel <ffmpeg-devel at ffmpeg.org>
>Envoyé : jeudi 5 décembre 2024 15:15
>I strongly object to using a resampler here. We *never* covert audio or
>video in decoders from the native codec samplerate or pixel format.
>This includes Opus. Opus carries the original samplerate of the audio
>encoded into its extradata. We ignore it, whilst libopusfile does not. I've been
>on #opus for a long time, and the only thing I've ever heard about libopusfile's
>behaviour has been confusion and criticism from its users.
>
>You should expose this, and let users make an informed decision. Most users
>will have a resampler anyway, and they may be *particularly sensitive* about
>any potential latency you introduce, so applying mandatory resampler here is
>both heavy-handed and problematic.

I was not aware of this, thank you for your information.

Users:
I still think the situation is different with Dolby E: from a user's perspective,
only the "original" aes sample_rate "exists" at all.
See here:
https://forum.videohelp.com/threads/400100-Dolby-E-Decoding-FFmpeg
"It might be an FFmpeg bug, so parameter -ar 48000 is used below
to 'fix' this issue via resampling"

Internal sample_rate:
In some cases, the native decoder sample rate is an integer-rounded value,
so there is no way for the user to avoid a slight drift over time.

Design:
I am confused, maybe I missunderstood something, because when working
on this, I was inspired by opus/dec.c which actually uses a resampler.
So I thought there would be no hard objection about the technical design.

Latency:
is obviously a good point and in my case I add about 30/40ms which is much,
but I don't think there are "very low" latency scenarios with dolby_e which is
not a generic nor affordable codec, so it more about live sport events at most,
but no interactive talks or so, I would say.
Moreover, the "official" dolby delay indeed is 1 full video/audio frame to keep
synced with the start of the video, so no user should expect a low delay.

Requirement:
As I described in my cover later but maybe I was not clear,
the use of a resampler is absolutely necessary to fix multiple sync issues,
and the tuning of it is somewhat tricky, it is not just about sample rate conversion.
So, if the resampler is not included, it must be inserted somewhere downstream
with advanced fine tuned settings.

Anyway, I can try to go forward.
And I also read that nested decoders should be avoided.
But at the end, it would mean that the output of s337m would be a "string of packets"
with pts adjusted packet by packet and having discontinuities (small overlap and/or
big or small gaps) between each packet. Is it really feasible and desirable ?
Well, there are currently so many issues specific to s337m and it seems impossible to
me to map this in ffmpeg. The dolby_e decoder itself has no active maintainer.
Maybe it should be recorded that s337m/dolby_e is an experimental thing and has
to be handled in a separate broadcast fork.

Nicolas