[Ffmpeg-devel] New/extended filter API discussion

[Luca Abeni]

Hi Benjamin (and all),

On Thu, 2007-01-04 at 12:24 +0100, Benjamin Zores wrote:
[...]
> Here are a few more thoughts and concerns about the libavfilter design.
> 
> #1: it has to support filter chaining (i.e filter_a then filter_b
> isn't the same than filter_b then filter_a). One should create an
> empty filter chain where you can append/prepend filters. I don't know
> if inserting filter at a defined position of the chain makes much
> sense, seems overkill.
I agree, but in my opinion we should not limit libavfilter to "filter
chains", but we should allow "filter graphs" (meaning that a filter can
have more than 1 input, and more than 1 output).

> #2: one must be able to enable/disable filters at runtime.
I agree (even if this looks more like a property of single filters... In
my idea the filter graph is statically built during an initialization
phase, but some - or all - filters can be dynamically configured -
allowing to transform them in "short circuits", disabling them)

> #3: one must be able to append/prepend filters at runtime (think about
> post-proc, deinterlacing or simply MPlayer's OSD menu).
In my original idea, I considered the filter graph as static. But maybe
I overlooked something. Why would this dynamic append / prepend be
needed? I do not know mplayer, but couldn't post-proc, deinterlacing,
OSD, etc be supported by statically inserting the (disabled) filters at
program startup, and dynamically enabling them when needed?

> Thus, it's a matter of:
> 
> av_filter_chain_apply() {
>   ...
>   while (filter->next)
>     if (filter->state == enabled)
>       av_filter_apply (filter);
>   ...
> }
I had a different idea; let me try to describe it with an example. In my
opinion, libavfilter should support both a "push" and a "pull" model,
but let's consider the "push" case. Let's assume we have a simple filter
chain Fa ---> Fb ---> Fc.
A frame is passed to the first filter of the chain (Fa) by:
1) Requesting an AVFrame to it
2) Filling the AVFrame
3) Submitting the AVFrame
4) When I do not need the AVFrame's memory buffer, I can notify this
fact to Fa that will free it

When Fa is requested an AVFrame (step 1), it can request it to Fb (that
can request it to Fc, and so on), or can provide it to the caller by
itself.
When Fa receives an AVFrame (step 3), it can do some operations on it,
and then send it to Fb (if the AVFrame has not been previously requested
to Fb, it will be requested now).
When the caller notifies that a buffer is not needed anymore (step 4),
Fa can free it, or notify this fact to Fb (depending on who allocated
the memory)

So, I think a filter should provide these methods:
    int (*init)(AVFilterContext *fc);
    int (*config_or_setup)(AVFilterContext *fc, char *config_info);
    AVFrame *(*get_buffer)(AVFilterContext *fc, int i);
    int (*push_buffer)(AVFilterContext *fc, AVFrame *, int i);
    int (*release_buffer)(AVFilterContext *fc, AVFrame * int i);

The "int i" parameter in get_buffer(), push_buffer(), and
release_buffer() is needed for supporting multiple input filter and
means "I (the caller) am your input number i".

As a result, a filter graph would be applied as follows:
f = avf_get_buffer(first_filter_c, 0);
/* Fill the AVFrame */
...
avf_push_buffer(first_filter_c, f, 0);
avf_release_buffer(first_filter_c, f, 0)

(the last filter of the chain will put the filtered frame somewhere,
save it to file, display it, or invoke some callback to notify that the
frame is ready)


> #4 the filter registering API should be public. It will allows loading
> external av filters through .so via dlopen(). Consider that some media
> app require some av filter of their own to work or interact with the
> player. If the filter is not (yet or for any reason) part of ffmpeg,
> the player should be able to still register it. Otherwise, considering
> it won't be shipped with the distro players, the media app will have
> to shipp it's program with a customized version of the  player and
> ffmpeg which is a not recommended nor desired.
I think libavformat and libavcodec already provide something similar for
codecs and formats (at least, I have some applications in which I
register and use my own format that is not part of libavformat...). I
expect libavfilter to work in a similar way.
So, I expect
av_register_filter(AVFilter *f);
AVFilter *av_find_filter(const char *name);
int av_open_filter(AVFilterContext **fc, AVFilter *f, const char *parameters);
etc...

As you can see from this email my ideas are still confused, but I am
trying to formalize them and write a more clear and concrete proposal
that I will send in the next days.


			Thanks,
				Luca
-- 
_____________________________________________________________________________
Copy this in your signature, if you think it is important:
                               N O    W A R ! ! !