[FFmpeg-devel] [PATCH] lavfi: port mcdeint filter from libmpcodecs
Clément Bœsch
ubitux at gmail.com
Mon May 27 13:30:27 CEST 2013
On Mon, May 27, 2013 at 03:24:33AM +0200, Stefano Sabatini wrote:
> TODO: bump minor, update changelog
> ---
> configure | 1 +
> doc/filters.texi | 33 ++++++
> libavfilter/Makefile | 2 +
> libavfilter/allfilters.c | 1 +
> libavfilter/vf_mcdeint.c | 288 ++++++++++++++++++++++++++++++++++++++++++++++
> 5 files changed, 325 insertions(+)
> create mode 100644 libavfilter/vf_mcdeint.c
>
> diff --git a/configure b/configure
> index 8b2b286..6863c3e 100755
> --- a/configure
> +++ b/configure
> @@ -2149,6 +2149,7 @@ hqdn3d_filter_deps="gpl"
> hue_filter_deps="gpl"
> interlace_filter_deps="gpl"
> kerndeint_filter_deps="gpl"
> +mcdeint_filter_deps="avcodec gpl"
LICENSE file needs update as well
> movie_filter_deps="avcodec avformat"
> mp_filter_deps="gpl avcodec swscale inline_asm"
> mpdecimate_filter_deps="gpl avcodec"
> diff --git a/doc/filters.texi b/doc/filters.texi
> index 04c97f4..66a4088 100644
> --- a/doc/filters.texi
> +++ b/doc/filters.texi
> @@ -4781,6 +4781,39 @@ lutyuv=y='bitand(val, 128+64+32)'
> @end example
> @end itemize
>
> + at section mcdeint
> +
> +Apply motion-compensation deinterlacing.
> +
> +It needs one field per frame as input and must thus be used together
> +with tfields=1 or yadif=1/3 or equivalent.
> +
> +This filter accepts the following options:
> + at table @option
> + at item mode
> +
> +Set the deinterlacing mode.
> +It accepts one of the following values:
> + at table @samp
> + at item fast
> + at item medium
> + at item slow
> +use iterative motion estimation
> + at item extra_slow
> +like @samp{slow}, but use multiple reference frames.
> + at end table
> +Default value is @samp{fast}.
> +
> + at item parity
> +0 or 1 selects which field to use.
> +
> + at item qp
> +Set qp to be used by the encoder.
> +
> +Higher values should result in a smoother motion vector field but less
> +optimal individual vectors.
> + at end table
> +
> @section mp
>
> Apply an MPlayer filter to the input video.
> diff --git a/libavfilter/Makefile b/libavfilter/Makefile
> index fa601b5..77757cc 100644
> --- a/libavfilter/Makefile
> +++ b/libavfilter/Makefile
> @@ -9,6 +9,7 @@ FFLIBS-$(CONFIG_ASYNCTS_FILTER) += avresample
> FFLIBS-$(CONFIG_ATEMPO_FILTER) += avcodec
> FFLIBS-$(CONFIG_DECIMATE_FILTER) += avcodec
> FFLIBS-$(CONFIG_DESHAKE_FILTER) += avcodec
> +FFLIBS-$(CONFIG_MCDEINT_FILTER) += avcodec
> FFLIBS-$(CONFIG_MOVIE_FILTER) += avformat avcodec
> FFLIBS-$(CONFIG_MP_FILTER) += avcodec
> FFLIBS-$(CONFIG_PAN_FILTER) += swresample
> @@ -149,6 +150,7 @@ OBJS-$(CONFIG_LUT3D_FILTER) += vf_lut3d.o
> OBJS-$(CONFIG_LUT_FILTER) += vf_lut.o
> OBJS-$(CONFIG_LUTRGB_FILTER) += vf_lut.o
> OBJS-$(CONFIG_LUTYUV_FILTER) += vf_lut.o
> +OBJS-$(CONFIG_MCDEINT_FILTER) += vf_mcdeint.o
> OBJS-$(CONFIG_MP_FILTER) += vf_mp.o
> OBJS-$(CONFIG_MPDECIMATE_FILTER) += vf_mpdecimate.o
> OBJS-$(CONFIG_NEGATE_FILTER) += vf_lut.o
> diff --git a/libavfilter/allfilters.c b/libavfilter/allfilters.c
> index 0f2442d..ad7937c 100644
> --- a/libavfilter/allfilters.c
> +++ b/libavfilter/allfilters.c
> @@ -147,6 +147,7 @@ void avfilter_register_all(void)
> REGISTER_FILTER(LUT, lut, vf);
> REGISTER_FILTER(LUTRGB, lutrgb, vf);
> REGISTER_FILTER(LUTYUV, lutyuv, vf);
> + REGISTER_FILTER(MCDEINT, mcdeint, vf);
> REGISTER_FILTER(MP, mp, vf);
> REGISTER_FILTER(MPDECIMATE, mpdecimate, vf);
> REGISTER_FILTER(NEGATE, negate, vf);
> diff --git a/libavfilter/vf_mcdeint.c b/libavfilter/vf_mcdeint.c
> new file mode 100644
> index 0000000..5b8cf28
> --- /dev/null
> +++ b/libavfilter/vf_mcdeint.c
> @@ -0,0 +1,288 @@
> +/*
> + * Copyright (c) 2006 Michael Niedermayer <michaelni at gmx.at>
> + *
> + * FFmpeg is free software; you can redistribute it and/or modify
> + * it under the terms of the GNU General Public License as published by
> + * the Free Software Foundation; either version 2 of the License, or
> + * (at your option) any later version.
> + *
> + * FFmpeg is distributed in the hope that it will be useful,
> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> + * GNU General Public License for more details.
> + *
> + * You should have received a copy of the GNU General Public License along
> + * with FFmpeg; if not, write to the Free Software Foundation, Inc.,
> + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
> + */
> +
> +/*
> + Known Issues:
> +
/**
* @file
* Motion Estimation Deinterlacer
*
* Known Issues:
*
* - ....
?
> +* The motion estimation is somewhat at the mercy of the input, if the input
> + frames are created purely based on spatial interpolation then for example
> + a thin black line or another random and not interpolateable pattern
> + will cause problems.
> + Note: completly ignoring the "unavailable" lines during motion estimation
> + did not look any better, so the most obvious solution would be to improve
> + tfields or penalize problematic motion vectors ...
> +
> +* If non iterative ME is used then snow currently ignores the OBMC
> + window and as a result sometimes creates artifacts-
> +
> +* Only past frames are used, we should ideally use future frames too,
> + something like filtering the whole movie in forward and then
> + backward direction seems like a interresting idea but the current
> + filter framework is FAR from supporting such things.
> +
> +* Combining the motion compensated image with the input image also is
> + not as trivial as it seems, simple blindly taking even lines from
> + one and odd ones from the other does not work at all as ME/MC
> + sometimes simple has nothing in the previous frames which matches
> + the current. The current algorithm has been found by trial and error
> + and almost certainly can be improved...
> +*/
> +
> +#include "libavutil/opt.h"
> +#include "libavutil/pixdesc.h"
> +#include "libavcodec/avcodec.h"
> +#include "avfilter.h"
> +#include "formats.h"
> +#include "internal.h"
> +
> +enum MCDeintMode {
> + MCDEINT_MODE_FAST = 0,
> + MCDEINT_MODE_MEDIUM,
> + MCDEINT_MODE_SLOW,
> + MCDEINT_MODE_EXTRA_SLOW,
> + MCDEINT_MODE_NB,
> +};
> +
> +typedef struct {
> + const AVClass *class;
> + enum MCDeintMode mode;
> + int qp;
> + int parity;
> + AVPacket pkt;
> + AVCodecContext *enc_ctx;
> + AVFrame *frame;
> +} MCDeintContext;
> +
> +#define OFFSET(x) offsetof(MCDeintContext, x)
> +#define FLAGS AV_OPT_FLAG_VIDEO_PARAM|AV_OPT_FLAG_FILTERING_PARAM
> +#define CONST(name, help, val, unit) { name, help, 0, AV_OPT_TYPE_CONST, {.i64=val}, INT_MIN, INT_MAX, FLAGS, unit }
> +
> +static const AVOption mcdeint_options[] = {
> + { "mode", "set mode", OFFSET(mode), AV_OPT_TYPE_INT, {.i64=MCDEINT_MODE_FAST}, 0, MCDEINT_MODE_NB-1, FLAGS, .unit="mode" },
> + CONST("fast", "", MCDEINT_MODE_FAST, "mode"),
> + CONST("medium", "", MCDEINT_MODE_MEDIUM, "mode"),
> + CONST("slow", "", MCDEINT_MODE_SLOW, "mode"),
> + CONST("extra_slow", "", MCDEINT_MODE_EXTRA_SLOW, "mode"),
> +
nit: "" → NULL ?
> + { "parity", "set parity map", OFFSET(parity), AV_OPT_TYPE_INT, {.i64=-1}, -1, 1, FLAGS },
> + { "qp", "set qp", OFFSET(qp), AV_OPT_TYPE_INT, {.i64=1}, INT_MIN, INT_MAX, FLAGS },
> + { NULL }
> +};
> +
> +AVFILTER_DEFINE_CLASS(mcdeint);
> +
> +static int config_props(AVFilterLink *inlink)
> +{
> + AVFilterContext *ctx = inlink->dst;
> + MCDeintContext *mcdeint = ctx->priv;
> + int i, ret;
> + AVCodec *enc = avcodec_find_encoder(AV_CODEC_ID_SNOW);
> +
> + if (!enc) {
> + av_log(ctx, AV_LOG_ERROR, "Snow encoder is not enabled in libavcodec\n");
> + return AVERROR(EINVAL);
> + }
> +
> + for (i = 0; i < 3; i++) {
> + AVCodecContext *enc_ctx;
> + AVDictionary *opts = NULL;
> +
> + mcdeint->enc_ctx = avcodec_alloc_context3(enc);
> + if (!mcdeint->enc_ctx)
> + return AVERROR(ENOMEM);
> + enc_ctx = mcdeint->enc_ctx;
> + enc_ctx->width = inlink->w;
> + enc_ctx->height = inlink->h;
> + enc_ctx->time_base = (AVRational){1,25}; // meaningless
> + enc_ctx->gop_size = 300;
> + enc_ctx->max_b_frames = 0;
> + enc_ctx->pix_fmt = AV_PIX_FMT_YUV420P;
> + enc_ctx->flags = CODEC_FLAG_QSCALE | CODEC_FLAG_LOW_DELAY;
> + enc_ctx->strict_std_compliance = FF_COMPLIANCE_EXPERIMENTAL;
> + enc_ctx->global_quality = 1;
> + av_dict_set(&opts, "memc_only", "1", 0);
> + enc_ctx->me_cmp = enc_ctx->me_sub_cmp = FF_CMP_SAD; //SSE;
> + enc_ctx->mb_cmp = FF_CMP_SSE;
> +
> + switch (mcdeint->mode) {
> + case 3:
> + enc_ctx->refs= 3;
nit: space before '='
Also since there is no break, please use /* Fallthrough */ comments.
> + case 2:
> + enc_ctx->me_method = ME_ITER;
> + case 1:
> + enc_ctx->flags |= CODEC_FLAG_4MV;
> + enc_ctx->dia_size = 2;
> + case 0:
> + enc_ctx->flags |= CODEC_FLAG_QPEL;
> + }
> +
> + ret = avcodec_open2(enc_ctx, enc, &opts);
> + av_dict_free(&opts);
> + if (ret < 0)
> + return ret;
> + }
> + mcdeint->frame = avcodec_alloc_frame();
> + if (!mcdeint->frame)
> + return AVERROR(ENOMEM);
> +
> + return 0;
> +}
> +
> +static av_cold void uninit(AVFilterContext *ctx)
> +{
> + MCDeintContext *mcdeint = ctx->priv;
> +
> + if (mcdeint->enc_ctx) {
> + avcodec_close(mcdeint->enc_ctx);
> + av_freep(&mcdeint->enc_ctx);
> + }
> + avcodec_free_frame(&mcdeint->frame);
> +}
> +
> +static int query_formats(AVFilterContext *ctx)
> +{
> + static const enum PixelFormat pix_fmts[] = {
> + AV_PIX_FMT_YUV420P, AV_PIX_FMT_GRAY8, AV_PIX_FMT_NONE
> + };
> +
> + ff_set_common_formats(ctx, ff_make_format_list(pix_fmts));
> +
> + return 0;
> +}
> +
> +static int filter_frame(AVFilterLink *inlink, AVFrame *inpic)
> +{
> + MCDeintContext *mcdeint = inlink->dst->priv;
> + AVFilterLink *outlink = inlink->dst->outputs[0];
> + AVFrame *outpic, *frame_dec;
> + int x, y, i, ret, got_frame;
> +
> + outpic = ff_get_video_buffer(outlink, outlink->w, outlink->h);
> + if (!outpic) {
> + av_frame_free(&inpic);
> + return AVERROR(ENOMEM);
> + }
> + av_frame_copy_props(outpic, inpic);
> +
> + for (i = 0; i < 3; i++) {
> + mcdeint->frame->data[i] = inpic->data [i];
> + mcdeint->frame->linesize[i] = inpic->linesize[i];
> + }
> +
> + mcdeint->enc_ctx->me_cmp =
> + mcdeint->enc_ctx->me_sub_cmp = FF_CMP_SAD /*| (p->parity ? FF_CMP_ODD : FF_CMP_EVEN)*/;
> + mcdeint->frame->quality = mcdeint->qp * FF_QP2LAMBDA;
> +
> + av_init_packet(&mcdeint->pkt);
> + mcdeint->pkt.data = NULL; // packet data will be allocated by the encoder
> + mcdeint->pkt.size = 0;
> +
> + ret = avcodec_encode_video2(mcdeint->enc_ctx, &mcdeint->pkt,
> + mcdeint->frame, &got_frame);
> + if (ret < 0)
> + return ret;
> + frame_dec = mcdeint->enc_ctx->coded_frame;
> +
> + for (i = 0; i < 3; i++) {
> + int is_chroma = !!i;
> + int w = inlink->w >> is_chroma;
> + int h = inlink->h >> is_chroma;
Shouldn't this be FF_CEIL_RSHIFT()?
> + int fils = frame_dec->linesize[i];
> + int srcs = inpic->linesize[i];
> +
> + for (y = 0; y < h; y++) {
> + if ((y ^ mcdeint->parity) & 1) {
> + for (x = 0; x < w; x++) {
> + if ((x-2)+(y-1) * w >= 0 && (x+2)+(y+1)*w < w*h) { //FIXME either alloc larger images or optimize this
> + uint8_t *filp = &frame_dec->data[i][x + y*fils];
> + uint8_t *srcp = &inpic->data[i][x + y*srcs];
> + int diff0 = filp[-fils] - srcp[-srcs];
> + int diff1 = filp[+fils] - srcp[+srcs];
> + int spatial_score = FFABS(srcp[-srcs-1] - srcp[+srcs-1])
> + + FFABS(srcp[-srcs ] - srcp[+srcs ])
> + + FFABS(srcp[-srcs+1] - srcp[+srcs+1]) - 1;
> + int temp = filp[0];
> +
> +#define CHECK(j) \
> + { int score = FFABS(srcp[-srcs-1+(j)] - srcp[+srcs-1-(j)]) \
> + + FFABS(srcp[-srcs +(j)] - srcp[+srcs -(j)]) \
> + + FFABS(srcp[-srcs+1+(j)] - srcp[+srcs+1-(j)]); \
> + if (score < spatial_score) { \
> + spatial_score = score; \
> + diff0 = filp[-fils+(j)] - srcp[-srcs+(j)]; \
> + diff1 = filp[+fils-(j)] - srcp[+srcs-(j)];
> +
> + CHECK(-1) CHECK(-2) }} }}
> + CHECK( 1) CHECK( 2) }} }}
Ow. I believe you can do better. What about something like:
#define GET_SCORE(j) = FFABS(srcp[-srcs-1+(j)] - srcp[+srcs-1-(j)]) \
+ FFABS(srcp[-srcs +(j)] - srcp[+srcs -(j)]) \
+ FFABS(srcp[-srcs+1+(j)] - srcp[+srcs+1-(j)])
#define SET_NEW_VALUES(j) do { \
spatial_score = score; \
diff0 = filp[-fils+(j)] - srcp[-srcs+(j)]; \
diff1 = filp[+fils-(j)] - srcp[+srcs-(j)]; \
} while (0)
#define CHECK(a, b) do { \
int score = GET_SCORE(a); \
if (score < spatial_score) { \
SET_NEW_VALUES(a); \
score = GET_SCORE(b); \
if (score < spatial_score) \
SET_NEW_VALUES(b); \
} \
} while (0)
CHECK(-1, -2);
CHECK( 1, 2);
That's a bit more code but it removes the { } evilness.
Note: above code is completely untested.
> + if (diff0 + diff1 > 0)
> + temp -= (diff0 + diff1 - FFABS(FFABS(diff0) - FFABS(diff1)) /2)/2;
> + else
> + temp -= (diff0 + diff1 + FFABS(FFABS(diff0) - FFABS(diff1)) /2)/2;
> + filp[0] =
> + outpic->data[i][x + y*outpic->linesize[i]] = temp > 255U ? ~(temp>>31) : temp;
> + } else {
> + outpic->data[i][x + y*outpic->linesize[i]] = frame_dec->data[i][x + y*fils];
> + }
> + }
> + }
> + }
> + for (y = 0; y < h; y++) {
> + if (!((y ^ mcdeint->parity) & 1)) {
> + for (x = 0; x < w; x++) {
> + frame_dec->data[i][x + y*fils] =
> + outpic->data[i][x + y*outpic->linesize[i]] = inpic->data[i][x + y*srcs];
> + }
> + }
> + }
> + }
> + mcdeint->parity ^= 1;
> +
> + av_free_packet(&mcdeint->pkt);
> + av_frame_free(&inpic);
> + return ff_filter_frame(outlink, outpic);
> +}
> +
> +static const AVFilterPad mcdeint_inputs[] = {
> + {
> + .name = "default",
> + .type = AVMEDIA_TYPE_VIDEO,
> + .filter_frame = filter_frame,
> + .config_props = config_props,
> + },
> + { NULL }
> +};
> +
> +static const AVFilterPad mcdeint_outputs[] = {
> + {
> + .name = "default",
> + .type = AVMEDIA_TYPE_VIDEO,
> + },
> + { NULL }
> +};
> +
> +AVFilter avfilter_vf_mcdeint = {
> + .name = "mcdeint",
> + .description = NULL_IF_CONFIG_SMALL("Apply motion compenstaing deinterlacing."),
Typo in "compensating"
> + .priv_size = sizeof(MCDeintContext),
> + .uninit = uninit,
> + .query_formats = query_formats,
> +
> + .inputs = mcdeint_inputs,
> + .outputs = mcdeint_outputs,
> + .priv_class = &mcdeint_class,
> +};
No more comment from me. You might want to wait for some comment from
Michael though.
--
Clément B.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 490 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20130527/c36febf6/attachment.asc>
More information about the ffmpeg-devel
mailing list