[FFmpeg-devel] [RFC]] swscale modernization proposal

[Niklas Haas]

On Sat, 22 Jun 2024 21:52:42 +0200 Michael Niedermayer <michael at niedermayer.cc> wrote:
> On Sat, Jun 22, 2024 at 05:10:28PM +0200, Niklas Haas wrote:
> > On Sat, 22 Jun 2024 15:23:22 +0100 Andrew Sayers <ffmpeg-devel at pileofstuff.org> wrote:
> > > On Sat, Jun 22, 2024 at 03:13:34PM +0200, Niklas Haas wrote:
> > > [...]
> > > > 
> > > > ## Comments / feedback?
> > > > 
> > > > Does the above approach seem reasonable? How do people feel about introducing
> > > > a new API vs. trying to hammer the existing API into the shape I want it to be?
> > > > 
> > > > I've attached an example of what <avscale.h> could end up looking like. If
> > > > there is broad agreement on this design, I will move on to an implementation.
> > > 
> > > API users seem to have difficulty with this type of big change[[1],
> > > and doing the interface before the implementation means there's less
> > > reason for developers to switch while you're still looking for feedback.
> > > 
> > > What's the plan to bring them along?
> > 
> > Since SwsContext is entirely internal, we can continue providing the
> > current API on top of whatever internal abstractions we arrive at. As
> > a trivial example, sws_scale() can construct a temporary AVFrame based
> > on the provided information, and simply pass that to avscale_frame(). So
> > I don't think legacy API users are at risk, or pressure to switch,
> > unless they want access to *new* functionality.
> > 
> > For that, the harder step is moving from sws_scale() to
> > sws_scale_frame(). This is something API users can *already* do. By
> > contrast, moving from sws_scale_frame() to avscale_frame() should
> > hopefully be simple, since it just requires making sure the AVFrame is
> > correctly tagged. Usually, the flow is in the opposite direction - users
> > receive a correctly tagged AVFrame and manually forward this information
> > to the SwsContext. So, most of the time, moving to a fully AVFrame-based
> > API will result in deleting code, rather than adding it.
> > 
> > If we wanted to maximize the transition comfort, we should consider
> > re-using the sws_scale_frame() entrypoint directly. The main reason I am
> > hesitant to do this is because sws_scale_frame() is currently tied to
> > the stateful configuration of SwsContext, and mostly ignores the AVFrame
> > metadata. While changing that is possible, it possibly presents a bigger
> > API break than simply introducing a new function.
> 
> I agree we should keep using the same swscale.h header. It matches the library
> name thats installed (thats also what the user expects and what (s)he is used to),
> and its what users #include today.
> Also its not a audio? scaler so the A is confusing.
> 
> And sws_scale_frame() should be used obviously if thats as you say does
> "maximize the transition comfort"
> 
> Maybe simply adding an option for the library user to set the behavior
> (favour AVFrame properties vs initial properties)
> And then eventually deprecate and phase out the initial ones
> 
> The big advantage here is that we capture all users, noone stays on the old
> API. And the transition is also simpler, if its just a flag to flip for someone
> to try the new fully stateless system.

This could definitely work. We could then also eventually flip the
condition to where the new behavior becomes the default, and you need to
set a flag to *disable* it.

And eventually deprecate sws_init_context(), sws_setCoefficients() etc.
altogether and just have sws_alloc_context() + sws_scale_frame() be the
preferred front-ends.

I expect the actual amount of work to be similar; rather than taking
SwsContext and pulling everything high-level out into AVScaleContext, we
start with SwsContext and pull everything low-level out into separate
sub-contexts (e.g. one SwsScaleContext for each individual scaling
step).

> 
> thx
> 
> [...]
> -- 
> Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> 
> Complexity theory is the science of finding the exact solution to an
> approximation. Benchmarking OTOH is finding an approximation of the exact
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> https://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> 
> To unsubscribe, visit link above, or email
> ffmpeg-devel-request at ffmpeg.org with subject "unsubscribe".