[FFmpeg-devel] Writing filters
James Darnley
james.darnley at gmail.com
Sat Jun 14 15:29:06 CEST 2014
On 2014-06-11 15:02, James Darnley wrote:
> I appreciate any insights people can give. Anything I learn, I will try
> to add to existing docs.
Thank you all for your help so far. I've got two files to show you.
They are two simple filters with a fair few comments in to try to
explain the points I had trouble understanding. The comments represent
what I understand at present so I welcome any corrections and suggestions.
The first (vf_example) is based on vf_null. It doesn't do anything
(other than pass frames through) but does try to show how
request_frame() and filter_frame() should be used. I think I have made
it clear where a real filter would need to do more.
The second (avf_example) is based on how far I have progressed to the
filter I am trying to make. It is an audio->video filter that does get
audio in and put video frames out but it doesn't actually process the
audio samples. I wrote it into a new file, cleaned it up and added
comments. I haven't finished commenting everything I would like to in
this file but it is in a state in which I can show it.
There are a few FIXMEs and TODOs in both files. The FIXMEs generally
represent something about which I'm not quite clear. The TODOs mark
places that I think could do with expanding.
I must confess that I haven't yet added them to ffmpeg and the build
system so I haven't actually tested them. Since the first is based on
vf_null it could probably just be dropped in and work. Since I have
added my filter to the build system, it does work so I would expect my
cleaned up version to do so as well. Though I have manually run gcc to
ensure that they compile.
In addition to looking at these two filters can I ask for a reply to the
email I sent on the 11th? I still had a few things I wasn't quite clear
on which are also expressed in some of the comments in these two files.
Thanks again.
-------------- next part --------------
/*
* This file is part of FFmpeg.
*
* FFmpeg is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* FFmpeg is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with FFmpeg; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
/**
* @file
* example audio->video filter
*/
#include "libavfilter/internal.h"
#include "libavutil/channel_layout.h"
#include "libavutil/opt.h"
typedef struct {
const AVClass *class;
int w, h;
AVRational rate;
int i;
int last_pts;
} ExampleContext;
#define OFFSET(x) offsetof(ExampleContext, x)
#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
static const AVOption example_options[] = {
{ "size", "video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = "vga"}, 0, 0, FLAGS },
{ "s", "video size", OFFSET(w), AV_OPT_TYPE_IMAGE_SIZE, {.str = "vga"}, 0, 0, FLAGS },
{ "rate", "video rate", OFFSET(rate), AV_OPT_TYPE_VIDEO_RATE, {.str = "30"}, 0, 0, FLAGS },
{ "r", "video rate", OFFSET(rate), AV_OPT_TYPE_VIDEO_RATE, {.str = "30"}, 0, 0, FLAGS },
{ NULL }
};
AVFILTER_DEFINE_CLASS(example);
static int query_formats(AVFilterContext *ctx)
{
AVFilterFormats *formats;
AVFilterChannelLayouts *layouts;
AVFilterLink *inlink = ctx->inputs[0];
AVFilterLink *outlink = ctx->outputs[0];
/* FIXME: cover when to check which formats/etc have been selected and what
* to do if properties change in the middle of a stream. */
/* FIXME: does the order of the lists express some sort of preference about
* formats? */
/* This is the list of sample formats the filter accepts as input. The list
* must be terminated by AV_SAMPLE_FMT_NONE. */
static const enum AVSampleFormat sample_fmts[] = {
AV_SAMPLE_FMT_S16,
AV_SAMPLE_FMT_S16P, /* This format is signed 16-bit samples with each
* channel separated into a different data plane
* within an AVFrame. This can allow some features
* to process data more quickly than having samples
* from each channel interleaved. */
AV_SAMPLE_FMT_NONE
};
/* This is the list of pixel formats the filter can output. The list must
* be terminated by AV_PIX_FMT_NONE. */
static const enum AVPixelFormat pixel_fmts[] = {
AV_PIX_FMT_GRAY8,
AV_PIX_FMT_NONE
};
/* This is the list of channel layouts the filter accepts. The list must be
* terminated by -1 (negative one). */
static const int64_t channel_layouts[] = {
AV_CH_LAYOUT_STEREO,
AV_CH_LAYOUT_MONO,
-1
};
/* You need to set what sample formats your filter supports. Most filters
* can probably work with many formats, you will just need to think about
* correctly designing the parts of the filter that do the actual
* processing. Do not accept more formats than your filter can actually
* work with. libavfilter will automatically convert to what is needed and
* can probably do it faster. */
formats = ff_make_format_list(sample_fmts);
if (!formats)
return AVERROR(ENOMEM);
ff_formats_ref(formats, &inlink->out_formats);
/* You need to set what channel layout(s) your filter requires. You should
* only accept the layouts your filter can actually work with rather than
* trying to convert them yourself. Again, libavfilter converts
* automatically. */
layouts = avfilter_make_format64_list(channel_layouts);
if (!layouts)
return AVERROR(ENOMEM);
ff_channel_layouts_ref(layouts, &inlink->out_channel_layouts);
/* You also need to set what sample rates your filter requires. A few
* filters will have a limited few rates they can work with, af_earwax for
* example, but most can accept arbitrary rates. This function below is a
* simple way to state that. Again you should only set which rates the
* filter actally requires because libavfilter will so the resampling
* quicker and with more quality. */
formats = ff_all_samplerates();
if (!formats)
return AVERROR(ENOMEM);
ff_formats_ref(formats, &inlink->out_samplerates);
/* This filter outputs video so it needs to set what pixel format it creates. */
/* TODO: expand this point. */
formats = ff_make_format_list(pixel_fmts);
if (!formats)
return AVERROR(ENOMEM);
ff_formats_ref(formats, &outlink->in_formats);
return 0;
}
/* After format negotiation has happened with query_formats you can set other
* properties of the input and output links. For most real filters you will
* need to add a function that operates on the input link to your filter. This
* filter does not have one because it doesn't use the input data. */
/* FIXME: is this where we can check which formats have been selected? */
/* FIXME: are these functions also called when stream propeties change? */
static int configure_output_properties(AVFilterLink *outlink)
{
ExampleContext *exctx = outlink->src->priv;
/* This filter creates video so it sets the dimentions that the user has
* selected. */
outlink->w = exctx->w;
outlink->h = exctx->h;
outlink->sample_aspect_ratio = (AVRational){1,1};
/* It is important to set timestamps and timebases correctly otherwise you
* cannot synchonise the video and audio streams. Here the filter sets the
* frame rate that the user has requested. */
outlink->frame_rate = exctx->rate;
/* The filter sets the timebase to the inverse of the frame rate allowing
* the timestamp on every frame to increase by 1. Other values may mean
* that you have to carefully calculate the timestamp. Leaving it unset
* means that the timebase of the input link is used. */
/* FIXME: is that actually correct? */
outlink->time_base = av_inv_q(exctx->rate);
return 0;
}
static int compare_pts(AVRational a_time_base, int64_t a_pts,
AVRational b_time_base, int64_t b_pts)
{
double a_time = a_pts * av_q2d(a_time_base);
double b_time = b_pts * av_q2d(b_time_base);
if (a_time > b_time)
return 1;
if (a_time < b_time)
return -1;
return 0;
}
static int filter_frame(AVFilterLink *inlink, AVFrame *insamples)
{
AVFilterContext *ctx = inlink->dst;
AVFilterLink *outlink = ctx->outputs[0];
ExampleContext *exctx = ctx->priv;
AVFrame *outpic;
int i, ret;
do {
outpic = ff_get_video_buffer(outlink, exctx->w, exctx->h);
if (!outpic)
return AVERROR(ENOMEM);
for (i = 0; i < exctx->h; i++)
memset(outpic->data[0] + i * outpic->linesize[0], (i == exctx->i)?255:0, exctx->w);
outpic->pts = exctx->last_pts++;
exctx->i = (exctx->i + 1) % exctx->h;
ret = ff_filter_frame(outlink, outpic);
} while (compare_pts(outlink->time_base, exctx->last_pts,
inlink->time_base, insamples->pts) < 0 && ret >= 0);
av_frame_free(&insamples);
return ret;
}
static const AVFilterPad example_inputs[] = {
{
.name = "default",
.type = AVMEDIA_TYPE_AUDIO,
.filter_frame = filter_frame,
},
{ NULL}
};
static const AVFilterPad example_outputs[] = {
{
.name = "default",
.type = AVMEDIA_TYPE_VIDEO,
.config_props = configure_output_properties,
},
{ NULL }
};
AVFilter ff_avf_example = {
.name = "example",
.description = NULL_IF_CONFIG_SMALL("Example filter which just draws a line moving steadily down the frame."),
.query_formats = query_formats,
.inputs = example_inputs,
.outputs = example_outputs,
.priv_size = sizeof(ExampleContext),
.priv_class = &example_class,
};
-------------- next part --------------
/*
* This file is part of FFmpeg.
*
* FFmpeg is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* FFmpeg is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with FFmpeg; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
/**
* @file
* example video filter
*/
#include "libavfilter/internal.h"
static int frames_still_needed(void)
{
return 0;
}
static int frames_to_output(void)
{
return 1;
}
static int filter_frame(AVFilterLink *inlink, AVFrame *in)
{
/* FIXME: how much do I need to explain about clean, readable code? */
int i, ret;
/* The first thing that all filters will want is a reference to their
* context structure. This filter doesn't have one because it does not have
* any options and it does not need to store any variables between
* functions. */
/* Most filters also want a reference to the output AVFilterLink. */
AVFilterLink *outlink = inlink->dst->outputs[0];
/* You will probably need to create an AVFrame into which the filter will
* write the results of its processing. */
/* TODO: cover writeable input AVFrames. */
AVFrame *out;
/* Some filters will require more than one input frame before they can
* perform their processing. If you need more data, whether it is video
* frames, audio samples (or perhaps both), you should store the data now
* and then return success. */
if (frames_still_needed())
return 0;
/* Because this is a basic example of what a filter should look like, it
* doesn't do any processing. It doesn't even bother creating a new frame. */
out = in;
/* After the filter has finished processing, you can pass the finished frame
* into the rest of the filter chain. A filter may produce more than one
* frame at any time. */
for (i = frames_to_output(); i > 0; i--) {
ret = ff_filter_frame(outlink, out);
}
return ret;
}
static int request_frame(AVFilterLink *outlink)
{
AVFilterLink *inlink = outlink->src->inputs[0];
int ret;
/* If a filter needs to request more frames it should be done here in the
* request_frame function. The return made in filter_frame goes back
* through the filter chain until ff_request_frame returns here. Then
* another frame can be requested (and so on) until a frame gan be generated
* in filter_frame and be sent to the output. */
do {
ret = ff_request_frame(inlink);
} while (frames_still_needed() && ret != AVERROR_EOF);
return ret;
}
static const AVFilterPad example_inputs[] = {
{
.name = "default",
.type = AVMEDIA_TYPE_VIDEO,
.filter_frame = filter_frame,
},
{ NULL }
};
static const AVFilterPad example_outputs[] = {
{
.name = "default",
.type = AVMEDIA_TYPE_VIDEO,
.request_frame = request_frame,
},
{ NULL }
};
AVFilter ff_vf_example = {
.name = "example",
.description = NULL_IF_CONFIG_SMALL("Pass the source unchanged to the output."),
.inputs = example_inputs,
.outputs = example_outputs,
};
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 683 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140614/66df5590/attachment.asc>
More information about the ffmpeg-devel
mailing list