[FFmpeg-devel] Discuss: Improved hardware acceleration support

[wm4]

On Fri, 31 Oct 2014 19:06:28 +0100
Hendrik Leppkes <h.leppkes at gmail.com> wrote:

> On Fri, Oct 31, 2014 at 5:59 PM, wm4 <nfxjfg at googlemail.com> wrote:
> 
> > On Thu, 30 Oct 2014 20:31:17 -0500
> > Rodger Combs <rodger.combs at gmail.com> wrote:
> >
> > > libavcodec currently has support for hardware-accelerated decoding, but
> > no support for encoding, and libavcodec+libavfilter+ffmpeg provide no
> > support for a decode->filter->encode pipeline that doesn't involve copying
> > buffers back and forth from the video card and cutting out a significant
> > amount of the gain provided by using hardware acceleration to begin with.
> > It'd be useful to provide a way to leave buffers on the GPU when possible,
> > and copy back and forth only when using a filter that can't be done on the
> > GPU.
> > > Some filters could even be run without copying back and forth; for
> > instance: scaling (for some scalers), overlays, cropping,
> > drawtext/subtitles (the drawing component, anyway), deinterlacing, trim,
> > and some post-processing could likely be done for a number of GPUs
> > relatively easily, and others could likely also be done with additional
> > work.
> > > This would probably require significant changes to AVFrame, various
> > lavc/lavfi structs and APIs, and ffmpeg.c, but it could likely produce
> > significant improvements in speed and power consumption when using systems
> > that can support a full decode->filter->encode pipeline on the GPU.
> > >
> > > Thoughts on feasibility and/or implementation details?
> >
> > It would be pretty simple.
> 
> 
> Really? :)
> 
> Every HW accel uses its own special format, how do you ever want to make
> lavfi be capable of handling those all? It would be a nightmare of HW-API
> specific stuff, and considering practically none of the software filtering
> code can be reused anyway, hardly worth trying to shoehorn that into lavfi
> anyway.
> All the pieces are already in place for an application to do this itself
> for the one or two HW APIs it cares about, however, just the generic case
> is an absolute nightmare.

Of course these filters would be highly API-specific. Unifying them
would probably not be possible on such a low level, and not all APIs
are equally flexible.

Whether it would be useful to add this to lavfi is another question.
Maybe it would - it could save the application programmer some
API-specific code, even if it still has to select and create the right
filters per API.