[FFmpeg-devel] [PATCH 6/6] lavfi: make AVFilterLink opaque in two major bumps.

[Nicolas George]

Le nonidi 29 frimaire, an CCXXV, Michael Niedermayer a écrit :
> though i wonder what mistake we did to end here, the original design
> was intended to be very simple ...

That was the mistake: trying for a simple design.

Ha ha! Only serious. Linear filter chains are simple, and could have
worked easily with a simple design. But non-linear filter graphs are
inherently more complex: the system needs to decide what filter to
activate first from several parallel branches.

Graphs with several inputs and/or outputs are even more difficult,
because the application needs to decide which input needs a frame most
urgently.

Both lavfi's original design (API and implementation) and its use in
ffmpeg (actually, avconv) for complex filter graphs neglect partially or
completely this issue.

I tried to address this, and it more or less work, but that was with
localized changes, not a real design. The changes became less localized
in ffmpeg, but still.

(Also, the code in ffmpeg is made more complex because of the "-c copy"
and subtitles paths, because they do not transit through lavfi, and for
copy not through lavc. I have ideas to make this simpler, I can share
them, but this is only remotely related to the current discussion and I
do not want to clutter it.)

> my interrest is having a stable public API and a plugin interface
> and iam interrested discussing that.
> Iam also interrested in discussing clearly defined problems,
> reproducable issues and solutions.
> about fine details i dont really care
> and your libavfilter patches are sometimes complex making it not a
> trivial quick thing to discuss them

"make filter_frame non-recursive" was the Beast. The next patches will
be much simpler and straightforward, mostly isolating existing code in
cleaner functions.

> You complain that noone else is working on libavfilter
> how many people know what _NEEDS_ to be done to make the API good
> enough for a first public and stable API ?
> 
> while i have a long todo and maybe no time for this, as it is i cant
> even work on making libavfilter public+stable. Its not a technical
> issue, its one of peoples requirements on what they require to be
> changed first.

This is true, but there is another side to the human problem: explaining
the plans and requirements of the API takes time; if nobody will read it
carefully, that time is better spent actually coding (I am writing this
mail by snippets while FATE is running).

Here is an outline, in approximate order (there are dependencies, but
not everything):

- Add a callback AVFilter.activate() to replace filter_frame() on all
  inputs and request_frame() on all outputs. Most non-trivial filters
  are written that way in the first place.

- Change buffersink to implement that callback and peek directly in the
  FIFO.

- Write a generic activate callback for 1-to-1 filters and use it.

- Rewrite framesync (the utility system for filters with several video
  inputs that need synchroized frames) to implement activate and use the
  FIFO directly.

- Allow to give buffersrc a timestamp on EOF, make sure the timestamp is
  forwarded by most filters and allow to retrieve it from buffersink.

  (If somebody has a suggestion of a good data structure for that...)

- Allow to set a callback on buffersinks and buffersrcs to be notified
  when a frame arrives or is needed. It is much more convenient than
  walking the buffers to check one by one.

- Allow to merge several filter graphs into one. This may be useful when
  applications have several graphs to run simultaneously, so that they
  do not need to decide which one to activate. Another option would be
  to have the functions in the next step work on possibly several
  graphs.

- Keep a priority queue of ready filters and use it instead of walking
  the graph.

- Add a function to run the graph until "something" happens; "something"
  meaning a stop instruction called by the callbacks.

- Add utility functions to easily run the graph. The most useful IMHO:
  run until a frame comes out from any sink (we get the frame and
  something useful identifying the sink) or one is needed from a source
  (we get something identifying the source) or EOF.

I think that covers it. Please feel free to ask for details, give
suggestions, etc. Please.

Regards,

-- 
  Nicolas George
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20161220/f4051340/attachment.sig>