[FFmpeg-devel] [PATCH] avfilter/vf_colorkey: Add colorkey video filter
Clément Bœsch
u at pkh.me
Tue Apr 21 04:50:58 CEST 2015
On Sun, Apr 19, 2015 at 06:15:48PM +0200, Timo Rothenpieler wrote:
> ---
> Changelog | 1 +
> MAINTAINERS | 1 +
> doc/filters.texi | 39 ++++++++
> libavfilter/Makefile | 1 +
> libavfilter/allfilters.c | 1 +
> libavfilter/vf_colorkey.c | 226 ++++++++++++++++++++++++++++++++++++++++++++++
> 6 files changed, 269 insertions(+)
> create mode 100644 libavfilter/vf_colorkey.c
>
> diff --git a/Changelog b/Changelog
> index 2e30b24..e99d1e9 100644
> --- a/Changelog
> +++ b/Changelog
> @@ -17,6 +17,7 @@ version <next>:
> - WebM Live Chunk Muxer
> - nvenc level and tier options
> - chorus filter
> +- colorkey video filter
>
>
> version 2.6:
> diff --git a/MAINTAINERS b/MAINTAINERS
> index 598c0b3..18f6ba3 100644
> --- a/MAINTAINERS
> +++ b/MAINTAINERS
> @@ -352,6 +352,7 @@ Filters:
> avf_showcqt.c Muhammad Faiz
> vf_blend.c Paul B Mahol
> vf_colorbalance.c Paul B Mahol
> + vf_colorkey.c Timo Rothenpieler
> vf_dejudder.c Nicholas Robbins
> vf_delogo.c Jean Delvare (CC <khali at linux-fr.org>)
> vf_drawbox.c/drawgrid Andrey Utkin
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 0f2e720..79c0753 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -3026,6 +3026,45 @@ colorbalance=rs=.3
> @end example
> @end itemize
>
> + at section colorkey
> +RGB colorspace color keying.
> +
> +The filter accepts the following options:
> +
> + at table @option
> + at item color
> +The color which will be replaced with transparency.
> +
> + at item similarity
> +Similarity percentage with the key color.
> +
> +0.01 matches only the exact key color, while 1.0 matches everything.
> +
This should probably be a "threshold" option.
> + at item blend
> +Blend percentage.
> +
> +0.0 makes pixels either fully transparent, or not transparent at all.
> +
> +Higher values result in semi-transparent pixels, with a higher transparency
> +the more similar the pixels color is to the key color.
> + at end table
> +
Shouldn't this blending be different near cutting zones (to smooth a bit
the overlay)?
We will also probably want some kind of greedy mode to enlarge the
selection, and options such as ignore very small zones (to avoid random
glitches). But anyway, this can obviously come later.
> + at subsection Examples
> +
> + at itemize
> + at item
> +Make every green pixel in the input image transparent:
> + at example
> +ffmpeg -i input.png -vf colorkey=green out.png
> + at end example
> +
> + at item
> +Make every red pixel in the input image transparent, with a high similarity tolerance:
> + at example
> +ffmpeg -i input.png -vf colorkey=0xFF0000:0.3 out.png
> + at end example
> + at end itemize
> +
I suppose the main usage is to overlay the patchwork onto some other
video, so maybe a complete example with an overlay filter might make
sense.
> @section colorlevels
>
> Adjust video input frames using levels.
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index 48cee50..60e6b52 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -102,6 +102,7 @@ OBJS-$(CONFIG_BOXBLUR_FILTER) += vf_boxblur.o
> OBJS-$(CONFIG_CODECVIEW_FILTER) += vf_codecview.o
> OBJS-$(CONFIG_COLORBALANCE_FILTER) += vf_colorbalance.o
> OBJS-$(CONFIG_COLORCHANNELMIXER_FILTER) += vf_colorchannelmixer.o
> +OBJS-$(CONFIG_COLORKEY_FILTER) += vf_colorkey.o
> OBJS-$(CONFIG_COLORLEVELS_FILTER) += vf_colorlevels.o
> OBJS-$(CONFIG_COLORMATRIX_FILTER) += vf_colormatrix.o
> OBJS-$(CONFIG_COPY_FILTER) += vf_copy.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 7961dca..2eeec7f 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -118,6 +118,7 @@ void avfilter_register_all(void)
> REGISTER_FILTER(CODECVIEW, codecview, vf);
> REGISTER_FILTER(COLORBALANCE, colorbalance, vf);
> REGISTER_FILTER(COLORCHANNELMIXER, colorchannelmixer, vf);
> + REGISTER_FILTER(COLORKEY, colorkey, vf);
> REGISTER_FILTER(COLORLEVELS, colorlevels, vf);
> REGISTER_FILTER(COLORMATRIX, colormatrix, vf);
> REGISTER_FILTER(COPY, copy, vf);
> diff --git a/libavfilter/vf_colorkey.c b/libavfilter/vf_colorkey.c
> new file mode 100644
> index 0000000..b5180cf
> --- /dev/null
> +++ b/libavfilter/vf_colorkey.c
> @@ -0,0 +1,226 @@
> +/*
> + * Copyright (c) 2015 Timo Rothenpieler <timo at rothenpieler.org>
> + *
> + * This file is part of FFmpeg.
> + *
> + * FFmpeg is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU Lesser General Public
> + * License as published by the Free Software Foundation; either
> + * version 2.1 of the License, or (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> + * Lesser General Public License for more details.
> + *
> + * You should have received a copy of the GNU Lesser General Public
> + * License along with FFmpeg; if not, write to the Free Software
> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> + */
> +
> +#include "libavutil/opt.h"
> +#include "avfilter.h"
> +#include "formats.h"
> +#include "internal.h"
> +#include "video.h"
> +
> +typedef struct ColorkeyContext {
> + const AVClass *class;
> +
> + uint8_t colorkey_rgba[4];
> + float cf[3];
normalized color factors? This 2-letter array is fine but can you add a
comment?
> +
> + float similarity;
> + float blend;
> +} ColorkeyContext;
> +
> +static inline int convert_format(int fmt)
nit: inline is probably useless since there is a single call and the
function is static, so it will be inlined anyway.
> +{
> + switch(fmt) {
> + case AV_PIX_FMT_0RGB:
> + return AV_PIX_FMT_ARGB;
> + case AV_PIX_FMT_0BGR:
> + return AV_PIX_FMT_ABGR;
> + case AV_PIX_FMT_BGR0:
> + return AV_PIX_FMT_BGRA;
> + case AV_PIX_FMT_RGB0:
> + return AV_PIX_FMT_RGBA;
> + default:
> + return fmt;
> + }
> +}
> +
> +static inline int offset_r(int fmt)
> +{
> + switch(fmt) {
> + case AV_PIX_FMT_ARGB:
> + return 1;
> + case AV_PIX_FMT_ABGR:
> + return 3;
> + case AV_PIX_FMT_BGRA:
> + return 2;
> + case AV_PIX_FMT_RGBA:
> + default:
> + return 0;
> + }
> +}
> +
> +static inline int offset_g(int fmt)
> +{
> + switch(fmt) {
> + case AV_PIX_FMT_ARGB:
> + return 2;
> + case AV_PIX_FMT_ABGR:
> + return 2;
> + case AV_PIX_FMT_BGRA:
> + return 1;
> + case AV_PIX_FMT_RGBA:
> + default:
> + return 1;
> + }
> +}
> +
> +static inline int offset_b(int fmt)
> +{
> + switch(fmt) {
> + case AV_PIX_FMT_ARGB:
> + return 3;
> + case AV_PIX_FMT_ABGR:
> + return 1;
> + case AV_PIX_FMT_BGRA:
> + return 0;
> + case AV_PIX_FMT_RGBA:
> + default:
> + return 2;
> + }
> +}
> +
> +static inline int offset_a(int fmt)
> +{
> + switch(fmt) {
> + case AV_PIX_FMT_ARGB:
> + return 0;
> + case AV_PIX_FMT_ABGR:
> + return 0;
> + case AV_PIX_FMT_BGRA:
> + return 3;
> + case AV_PIX_FMT_RGBA:
> + default:
> + return 3;
> + }
> +}
> +
> +#define SATURATE(val) (FFMAX(FFMIN((val), 1.0), 0.0))
> +
You have clip functions for that, see av_clip*
> +static uint8_t do_colorkey_pixel(ColorkeyContext *ctx, uint8_t r, uint8_t g, uint8_t b)
> +{
> + float diff_r = r / 255.0 - ctx->cf[0];
> + float diff_g = g / 255.0 - ctx->cf[1];
> + float diff_b = b / 255.0 - ctx->cf[2];
> +
> + float diff = sqrt(diff_r * diff_r + diff_g * diff_g + diff_b * diff_b);
> +
Unless I'm mistaken, you can avoid a bunch of division, by simply doing:
const int dr = X - ctx->colorkey_rgba[0];
const int dg = X - ctx->colorkey_rgba[1];
const int db = X - ctx->colorkey_rgba[2];
sqrt((dr*dr + dg*dg + db*db) / 255.)
> + return SATURATE(FFMAX(diff - ctx->similarity, 0.0) / ctx->blend) * 255.0;
If you scale instead similarity to u8, can't you avoid the need for float
altogether?
Making it bitexact might make it faster, will probably allow easier SIMD,
and will make possible to add a FATE test.
> +}
> +
> +static void do_colorkey(ColorkeyContext *ctx, AVFrame *frame)
> +{
> + int o_r, o_g, o_b, o_a;
> + int o, x, y;
> +
> + frame->format = convert_format(frame->format);
> +
> + o_r = offset_r(frame->format);
> + o_g = offset_g(frame->format);
> + o_b = offset_b(frame->format);
> + o_a = offset_a(frame->format);
> +
Mmh... can't you use ff_fill_rgba_map (drawutils)?
> + for (x = 0; x < 3; ++x)
> + ctx->cf[x] = ctx->colorkey_rgba[x] / 255.0;
> +
> + for (y = 0; y < frame->height; ++y) {
> + for (x = 0; x < frame->width; ++x) {
nit: you can avoid this ++c scheme, it makes no sense in C
> + o = frame->linesize[0] * y + x * 4;
> +
> + frame->data[0][o + o_a] =
> + do_colorkey_pixel(ctx,
> + frame->data[0][o + o_r],
> + frame->data[0][o + o_g],
> + frame->data[0][o + o_b]);
> + }
Note: seems very easy to add threading. See doc/writing_filters.txt
You can also do inplace processing if the frame is writable (refer to the
same doc for more info)
> + }
> +}
> +
> +static AVFrame *get_video_buffer(AVFilterLink *link, int w, int h)
> +{
> + AVFrame *picref = ff_default_get_video_buffer(link, w, h);
> + do_colorkey(link->dst->priv, picref);
> + return picref;
> +}
> +
You don't need this.
> +static int filter_frame(AVFilterLink *link, AVFrame *inpicref)
> +{
> + do_colorkey(link->dst->priv, inpicref);
nit: maybe just inline that function yourself here
note: inpicref is a deprecated terminology, you should use "frame" or
"inframe"
> + return ff_filter_frame(link->dst->outputs[0], inpicref);
> +}
> +
> +static int query_formats(AVFilterContext *ctx)
> +{
> + static const enum AVPixelFormat pixel_fmts_eq[] = {
> + AV_PIX_FMT_0RGB,
> + AV_PIX_FMT_ARGB,
> + AV_PIX_FMT_RGB0,
> + AV_PIX_FMT_RGBA,
> + AV_PIX_FMT_0BGR,
> + AV_PIX_FMT_ABGR,
> + AV_PIX_FMT_BGR0,
> + AV_PIX_FMT_BGRA,
> + AV_PIX_FMT_NONE
> + };
> +
> + AVFilterFormats *fmts_list = ff_make_format_list(pixel_fmts_eq);
> + if (!fmts_list)
> + return AVERROR(ENOMEM);
> +
> + return ff_set_common_formats(ctx, fmts_list);
> +}
> +
> +static const AVFilterPad colorkey_inputs[] = {
> + {
> + .name = "default",
> + .type = AVMEDIA_TYPE_VIDEO,
> + .get_video_buffer = get_video_buffer,
> + .filter_frame = filter_frame,
> + },
> + { NULL }
> +};
> +
> +static const AVFilterPad colorkey_outputs[] = {
> + {
> + .name = "default",
> + .type = AVMEDIA_TYPE_VIDEO,
> + },
> + { NULL }
> +};
> +
> +#define OFFSET(x) offsetof(ColorkeyContext, x)
> +#define FLAGS AV_OPT_FLAG_FILTERING_PARAM|AV_OPT_FLAG_VIDEO_PARAM
> +
> +static const AVOption colorkey_options[] = {
> + { "color", "set the colorkey key color", OFFSET(colorkey_rgba), AV_OPT_TYPE_COLOR, { .str = "black" }, CHAR_MIN, CHAR_MAX, FLAGS },
> + { "similarity", "set the colorkey similarity value", OFFSET(similarity), AV_OPT_TYPE_FLOAT, { .dbl = 0.01 }, 0.01, 1.0, FLAGS },
> + { "blend", "set the colorkey key blend value", OFFSET(blend), AV_OPT_TYPE_FLOAT, { .dbl = 0.0 }, 0.0, 1.0, FLAGS },
> + { NULL }
> +};
> +
> +AVFILTER_DEFINE_CLASS(colorkey);
> +
> +AVFilter ff_vf_colorkey = {
> + .name = "colorkey",
> + .description = NULL_IF_CONFIG_SMALL("colorkey filter"),
> + .priv_size = sizeof(ColorkeyContext),
> + .priv_class = &colorkey_class,
> + .query_formats = query_formats,
> + .inputs = colorkey_inputs,
> + .outputs = colorkey_outputs,
> +};
--
Clément B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 473 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20150421/ffeec436/attachment.asc>
More information about the ffmpeg-devel
mailing list