[FFmpeg-devel] [Patch] [avfilter] refactor floating point based interpolation and introduce in vf_lenscorrection
Michael Niedermayer
michaelni at gmx.at
Wed Aug 20 11:47:05 CEST 2014
On Wed, Aug 20, 2014 at 08:14:31AM +0200, Daniel Oberhoff wrote:
>
>
> Von meinem iPhone gesendet
>
> > Am 20.08.2014 um 03:16 schrieb Michael Niedermayer <michaelni at gmx.at>:
> >
> >> On Wed, Aug 20, 2014 at 12:12:49AM +0200, Daniel Oberhoff wrote:
> >> Hello,
> >>
> >> As a follow-up to my last patch I now factored out the floating point based interpolation from transform.h/transform.c and applied it in the vf_lenscorrection filter I introduced in my last patch. What I did not do is also factor out fixed point based interpolation as used in vf_rotate and vf_perspective and maybe others as could probably also be done. Also I did not look further for uses of floating point based interpolation. Basically I just wanted to introduce interpolation in vf_lenscorrection without code duplication. As a side note I also tried to introduce fixed point calculations to vf_lenscorrection but found myself effectively doing floating point “by hand” since due to the high order of the algorithm (up to 4th order) it is very hard to keep track of the right amount of pre/post-comma digits for a given step in the algorithm and given parameters and it felt rather futile after a while.
> >
> >> Looking forward to reviews :).
> >
> > why did you use the deshake code and not the vf_perspective code ?!
> > i suspect this will be quite a bit harder to get into shape for a
> > common API
> >
>
> As i Said, perspective does fixed point calculations. Also would you care to ekaborate what exactly are your whoes with this API?
there where further commments further down in the reply, that list
some problems
In addition, being float based is already a problem on its own,
we dont really want it to be float based. Its very likely slower and
its not bitexact making tests harder.
besides a API that works 1 sample at a time redundantly recalculates
the Coefficients for each color plane, also doing 1 sample per call
isnt very SIMD friendly
or to see this from another point of view
what we would like to have is a SSE*/AVX* based generic resampling
code using fixed point (with 16bit coeffs for 8bit samples for example)
Theres no need to implement SSE/AVX but the API should be a step
toward this, and the patch feels like its going the opposit direction
this is even worse as this is common API, so other filters could
start using this increasing the amount of code that would eventually
be changed/updated/rewritten
>
> >
> >>
> >> From 4b271f72946aeebf5603cc8779f6b61ff0c1bd49 Mon Sep 17 00:00:00 2001
> >> From: James Almer <jamrial at gmail.com>
> >> Date: Sun, 10 Aug 2014 02:24:01 -0300
> >> Subject: [PATCH] afvilter: re-factor/re-use floating point based interpolation
> >> from vf_perspective
> >>
> >> ---
> >> doc/filters.texi | 11 +++
> >> libavfilter/interpolate.h | 144 ++++++++++++++++++++++++++++++++++++++++
> >> libavfilter/transform.c | 91 ++-----------------------
> >> libavfilter/transform.h | 14 +---
> >> libavfilter/vf_lenscorrection.c | 21 ++++--
> >> 5 files changed, 176 insertions(+), 105 deletions(-)
> >> create mode 100644 libavfilter/interpolate.h
> >>
> >> diff --git a/doc/filters.texi b/doc/filters.texi
> >> index 0ca1d6f..2edefc4 100644
> >> --- a/doc/filters.texi
> >> +++ b/doc/filters.texi
> >> @@ -5582,6 +5582,17 @@ height.
> >> Coefficient of the quadratic correction term. 0.5 means no correction.
> >> @item k2
> >> Coefficient of the double quadratic correction term. 0.5 means no correction.
> >> + at item interpolation
> >> +Set the interpolation method for the transformation
> >> +
> >> +It accepts the following values:
> >> + at table @samp
> >> + at item nearest
> >> + at item linear
> >> + at item cubic
> >> + at end table
> >> +Default value is @samp{linear}.
> >> +
> >> @end table
> >>
> >> The formula that generates the correction is:
> >> diff --git a/libavfilter/interpolate.h b/libavfilter/interpolate.h
> >> new file mode 100644
> >> index 0000000..6f7a849
> >> --- /dev/null
> >> +++ b/libavfilter/interpolate.h
> >> @@ -0,0 +1,144 @@
> >> +/*
> >> + * Copyright (C) 2010 Georg Martius <georg.martius at web.de>
> >> + * Copyright (C) 2010 Daniel G. Taylor <dan at programmer-art.org>
> >> + *
> >> + * This file is part of FFmpeg.
> >> + *
> >> + * FFmpeg is free software; you can redistribute it and/or
> >> + * modify it under the terms of the GNU Lesser General Public
> >> + * License as published by the Free Software Foundation; either
> >> + * version 2.1 of the License, or (at your option) any later version.
> >> + *
> >> + * FFmpeg is distributed in the hope that it will be useful,
> >> + * but WITHOUT ANY WARRANTY; without even the implied warranty of
> >> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> >> + * Lesser General Public License for more details.
> >> + *
> >> + * You should have received a copy of the GNU Lesser General Public
> >> + * License along with FFmpeg; if not, write to the Free Software
> >> + * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
> >> + */
> >> +
> >> +#ifndef AVFILTER_INTERPOLATE_H
> >> +#define AVFILTER_INTERPOLATE_H
> >> +
> >> +enum InterpolateMethod {
> >> + INTERPOLATE_NEAREST, //< Nearest-neighbor (fast)
> >> + INTERPOLATE_BILINEAR, //< Bilinear
> >> + INTERPOLATE_BIQUADRATIC, //< Biquadratic (best)
> >> + INTERPOLATE_COUNT, //< Number of interpolation methods
> >> +};
> >> +
> >> +// Shortcuts for the fastest and best interpolation methods
> >> +#define INTERPOLATE_DEFAULT INTERPOLATE_BILINEAR
> >> +#define INTERPOLATE_FAST INTERPOLATE_NEAREST
> >> +#define INTERPOLATE_BEST INTERPOLATE_BIQUADRATIC
> >> +
> >> +#define INTERPOLATE_METHOD(name) \
> >> + static av_always_inline uint8_t name(float x, float y, const uint8_t *src, \
> >> + int width, int height, int stride, uint8_t def)
> >> +
> >
> >> +/**
> >> + * Nearest neighbor interpolation
> >> + */
> >> +INTERPOLATE_METHOD(interpolate_nearest)
> >> +{
> >> + if (x < 0 || x >= width || y < 0 || y >= height) {
> >> + return def;
> >> + } else {
> >> + return src[(int)(x + 0.5f) + stride * (int)(y + 0.5f)];
> >> + }
> >> +}
> >
> > i dont think using float in a nearest neighbor resampler is acceptable
> >
> >
> >> +
> >> +/**
> >> + * Bilinear interpolation
> >> + */
> >> +INTERPOLATE_METHOD(interpolate_bilinear)
> >> +{
> >> + int x_c, x_f, y_c, y_f;
> >> + int v1, v2, v3, v4;
> >> + const uint8_t *line_y_f, *line_y_c;
> >> +
> >> + if (x < 0 || x >= width || y < 0 || y >= height) {
> >> + return def;
> >> + } else {
> >> + x_f = (int)x;
> >> + x_c = x_f + 1;
> >> +
> >> + y_f = (int)y;
> >> + y_c = y_f + 1;
> >> +
> >> + line_y_f = src + stride * y_c;
> >> + line_y_c = line_y_f + stride;
> >> +
> >> + v1 = line_y_c[x_c];
> >> + v2 = line_y_f[x_c];
> >> + v3 = line_y_c[x_f];
> >> + v4 = line_y_f[x_f];
> >> +
> >> + return (v1*(x - x_f)*(y - y_f) + v2*((x - x_f)*(y_c - y)) +
> >> + v3*(x_c - x)*(y - y_f) + v4*((x_c - x)*(y_c - y)));
> >> + }
> >> +}
> >
> >
> >> +
> >> +/**
> >> + * Biquadratic interpolation
> >> + */
> >> +INTERPOLATE_METHOD(interpolate_biquadratic)
> >> +{
> >> + int x_c, x_f, y_c, y_f;
> >> + uint8_t v1, v2, v3, v4;
> >> + float f1, f2, f3, f4;
> >> + const uint8_t *line_y_f, *line_y_c;
> >> +
> >> + if (x < 0 || x >= width || y < 0 || y >= height) {
> >> + return def;
> >> + } else {
> >> + x_f = (int)x;
> >> + x_c = x_f + 1;
> >> + y_f = (int)y;
> >> + y_c = y_f + 1;
> >> +
> >> + line_y_f = src + stride * y_c;
> >> + line_y_c = line_y_f + stride;
> >> +
> >> + v1 = line_y_c[x_c];
> >> + v2 = line_y_f[x_c];
> >> + v3 = line_y_c[x_f];
> >> + v4 = line_y_f[x_f];
> >> +
> >> + f1 = 1 - sqrt((x_c - x) * (y_c - y));
> >> + f2 = 1 - sqrt((x_c - x) * (y - y_f));
> >> + f3 = 1 - sqrt((x - x_f) * (y_c - y));
> >> + f4 = 1 - sqrt((x - x_f) * (y - y_f));
> >> + return (v1 * f1 + v2 * f2 + v3 * f3 + v4 * f4) / (f1 + f2 + f3 + f4);
> >> + }
> >> +}
> >
> > There should be bilinear and bicubic filters.
> > If you want to add windowed sinc filters or spline based ones i dont
> > mind.
> > But ive no faith that this odd filter above which has the order of a
> > bilinear one will look all that great. And it sure wont be fast with
> > sqrt() per sample
> >
> > also
> > one way to do higher order filters is to create a LUT with the filter
> > Coefficients, for example if you want 8bit sub pixel precission and
> > a bicubic filter that makes just 256 * 4 entries, 4 Coefficients for
> > each 256 subpixel positions.
> >
> > That LUT can be filled by using the float code from vf_perspective
> >
> > or if done without LUT the code from vf_perspective can also be used
> > directly and doesnt need any functions like sqrt() and at the
> > same time should be higher quality
> >
> > [...]
> > --
> > Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
> >
> > Complexity theory is the science of finding the exact solution to an
> > approximation. Benchmarking OTOH is finding an approximation of the exact
> > _______________________________________________
> > ffmpeg-devel mailing list
> > ffmpeg-devel at ffmpeg.org
> > http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
> _______________________________________________
> ffmpeg-devel mailing list
> ffmpeg-devel at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/ffmpeg-devel
--
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
While the State exists there can be no freedom; when there is freedom there
will be no State. -- Vladimir Lenin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20140820/0a1190e4/attachment.asc>
More information about the ffmpeg-devel
mailing list