[FFmpeg-devel] [PATCHv2] add signature filter for MPEG7 video signature

Michael Niedermayer michael at niedermayer.cc
Sat Apr 16 05:16:04 CEST 2016


On Sat, Apr 16, 2016 at 12:07:27AM +0200, Gerion Entrup wrote:
> On Donnerstag, 14. April 2016 19:52:46 CEST Michael Niedermayer wrote:
> > On Thu, Apr 14, 2016 at 07:06:29PM +0200, Gerion Entrup wrote:
> > > On Montag, 11. April 2016 14:54:57 CEST Michael Niedermayer wrote:
> > > > On Mon, Apr 11, 2016 at 02:30:37PM +0200, Gerion Entrup wrote:
> > > > > On Montag, 11. April 2016 12:57:17 CEST Michael Niedermayer wrote:
> > > > > > On Mon, Apr 11, 2016 at 04:25:28AM +0200, Gerion Entrup wrote:
> > > > > > > On Donnerstag, 7. April 2016 00:35:25 CEST Michael Niedermayer wrote:
> > > > > > > > On Wed, Mar 30, 2016 at 11:02:36PM +0200, Gerion Entrup wrote:
> > > > > > > > > On Mittwoch, 30. März 2016 22:57:47 CEST Gerion Entrup wrote:
> > > > > > > > > > Add improved patch.
> > > > > > > > > 
> > > > > > > > > Rebased to master.
> > > > > > > > > 
> > > > > > > > >  Changelog                      |    1
> > > > > > > > >  configure                      |    1
> > > > > > > > >  doc/filters.texi               |   70 +++
> > > > > > > > >  libavfilter/Makefile           |    1
> > > > > > > > >  libavfilter/allfilters.c       |    1
> > > > > > > > >  libavfilter/signature.h        |  554
> > > > > > > > >  ++++++++++++++++++++++++++++++
> > > > > > > > >  libavfilter/signature_lookup.c |  550
> > > > > > > > >  ++++++++++++++++++++++++++++++
> > > > > > > > >  libavfilter/version.h          |    4
> > > > > > > > >  libavfilter/vf_signature.c     |  741
> > > > > > > > >  +++++++++++++++++++++++++++++++++++++++++ 9 files changed, 1921
> > > > > > > > >  insertions(+), 2 deletions(-)
> > > > > > > > > 
> > > > > > > > > 9192f27ded45c607996b4e266b6746f807c9a7fd 
> > > > > > > > > 0001-add-signature-filter-for-MPEG7-video-signature.patch From
> > > > > > > > > 9646ed6f0cf78356cf2914a60705c98d8f21fe8a Mon Sep 17 00:00:00 2001
> > > > > > > > > From: Gerion Entrup <gerion.entrup at flump.de>
> > > > > > > > > Date: Sun, 20 Mar 2016 11:10:31 +0100
> > > > > > > > > Subject: [PATCH] add signature filter for MPEG7 video signature
> > > > > > > > > 
> > > > > > > > > This filter does not implement all features of MPEG7. Missing
> > > > > > > > > features:
> > > > > > > > > - compression of signature files
> > > > > > > > > - work only on (cropped) parts of the video
> > > > > > > > > ---
> > > > > > > > > 
> > > > > > > > >  Changelog                      |   1 +
> > > > > > > > >  configure                      |   1 +
> > > > > > > > >  doc/filters.texi               |  70 ++++
> > > > > > > > >  libavfilter/Makefile           |   1 +
> > > > > > > > >  libavfilter/allfilters.c       |   1 +
> > > > > > > > >  libavfilter/signature.h        | 554
> > > > > > > > >  ++++++++++++++++++++++++++++++
> > > > > > > > >  libavfilter/signature_lookup.c | 550
> > > > > > > > >  ++++++++++++++++++++++++++++++
> > > > > > > > >  libavfilter/version.h          |   4 +-
> > > > > > > > >  libavfilter/vf_signature.c     | 741
> > > > > > > > >  +++++++++++++++++++++++++++++++++++++++++ 9 files changed, 1921
> > > > > > > > >  insertions(+), 2 deletions(-)
> > > > > > > > >  create mode 100644 libavfilter/signature.h
> > > > > > > > >  create mode 100644 libavfilter/signature_lookup.c
> > > > > > > > >  create mode 100644 libavfilter/vf_signature.c
> > > > > > > > > 
> > > > > > > > > diff --git a/Changelog b/Changelog
> > > > > > > > > index 7b0187d..8a2b7fd 100644
> > > > > > > > > --- a/Changelog
> > > > > > > > > +++ b/Changelog
> > > > > > > > > 
> > > > > > > > > @@ -18,6 +18,7 @@ version <next>:
> > > > > > > > >  - coreimage filter (GPU based image filtering on OSX)
> > > > > > > > >  - libdcadec removed
> > > > > > > > >  - bitstream filter for extracting DTS core
> > > > > > > > > 
> > > > > > > > > +- MPEG-7 Video Signature filter
> > > > > > > > > 
> > > > > > > > >  version 3.0:
> > > > > > > > >  - Common Encryption (CENC) MP4 encoding and decoding support
> > > > > > > > > 
> > > > > > > > > diff --git a/configure b/configure
> > > > > > > > > index e550547..fe29827 100755
> > > > > > > > > --- a/configure
> > > > > > > > > +++ b/configure
> > > > > > > > > @@ -2979,6 +2979,7 @@ showspectrum_filter_deps="avcodec"
> > > > > > > > > 
> > > > > > > > >  showspectrum_filter_select="fft"
> > > > > > > > >  showspectrumpic_filter_deps="avcodec"
> > > > > > > > >  showspectrumpic_filter_select="fft"
> > > > > > > > > 
> > > > > > > > > +signature_filter_deps="gpl avcodec avformat"
> > > > > > > > > 
> > > > > > > > >  smartblur_filter_deps="gpl swscale"
> > > > > > > > >  sofalizer_filter_deps="netcdf avcodec"
> > > > > > > > >  sofalizer_filter_select="fft"
> > > > > > > > > 
> > > > > > > > > diff --git a/doc/filters.texi b/doc/filters.texi
> > > > > > > > > index 5d6cf52..a95f5a7 100644
> > > > > > > > > --- a/doc/filters.texi
> > > > > > > > > +++ b/doc/filters.texi
> > > > > > > > > @@ -11559,6 +11559,76 @@ saturation maximum:
> > > > > > > > > %@{metadata:lavfi.signalstats.SATMAX@}> > > > > 
> > > > > > > > >  @end example
> > > > > > > > >  @end itemize
> > > > > > > > > 
> > > > > > > > > + at anchor{signature}
> > > > > > > > > + at section signature
> > > > > > > > > +
> > > > > > > > > +Calculates the MPEG-7 Video Signature. The filter could handle
> > > > > > > > > more than one +input. In this case the matching between the
> > > > > > > > > inputs could be calculated. The +filter passthrough the first
> > > > > > > > > input. The output is written in XML. +
> > > > > > > > > +It accepts the following options:
> > > > > > > > > +
> > > > > > > > > + at table @option
> > > > > > > > > + at item mode
> > > > > > > > > 
> > > > > > > > > +Enable the calculation of the matching. The option value must be
> > > > > > > > > 0 (to disable +or 1 (to enable). Optionally you can set the mode
> > > > > > > > > to 2. Then the detection ends, +if the first matching sequence it
> > > > > > > > > reached. This should be slightly faster. +Per default the
> > > > > > > > > detection is disabled.
> > > > > > > > 
> > > > > > > > these shuld probably support named identifers not (only) 0/1/2
> > > > > > > 
> > > > > > > done
> > > > > > 
> > > > > > it should use AV_OPT_TYPE_INT and AV_OPT_TYPE_CONST not a string
> > > > > > 
> > > > > > > > > +
> > > > > > > > > + at item nb_inputs
> > > > > > > > > +Set the number of inputs. The option value must be a non negative
> > > > > > > > > interger. +Default value is 1.
> > > > > > > > > +
> > > > > > > > > + at item filename
> > > > > > > > > +Set the path to witch the output is written. If there is more
> > > > > > > > > than one input, +the path must be a prototype, i.e. must contain
> > > > > > > > > %d or %0nd (where n is a positive +integer), that will be
> > > > > > > > > replaced with the input number. If no filename is +specified, no
> > > > > > > > > output will be written. This is the default.
> > > > > > > > > +
> > > > > > > > > 
> > > > > > > > > + at item xml
> > > > > > > > > +Choose the output format. If set to 1 the filter will write XML,
> > > > > > > > > if set to 0 +the filter will write binary output. The default is
> > > > > > > > > 0.
> > > > > > > > 
> > > > > > > > format=xml/bin/whatever
> > > > > > > > seems better as its more extensible
> > > > > > > 
> > > > > > > done
> > > > > > > 
> > > > > > > > > +
> > > > > > > > > + at item th_d
> > > > > > > > > +Set threshold to detect one word as similar. The option value
> > > > > > > > > must be an integer +greater than zero. The default value is 9000.
> > > > > > > > > +
> > > > > > > > > + at item th_dc
> > > > > > > > > +Set threshold to detect all words as similar. The option value
> > > > > > > > > must be an integer +greater than zero. The default value is
> > > > > > > > > 60000.
> > > > > > > > > +
> > > > > > > > > + at item th_xh
> > > > > > > > > +Set threshold to detect frames as similar. The option value must
> > > > > > > > > be an integer +greater than zero. The default value is 116.
> > > > > > > > > +
> > > > > > > > > + at item th_di
> > > > > > > > > +Set the minimum length of a sequence in frames to recognize it as
> > > > > > > > > matching
> > > > > > > > > +sequence. The option value must be a non negative integer value.
> > > > > > > > > +The default value is 0.
> > > > > > > > > +
> > > > > > > > > + at item th_it
> > > > > > > > > +Set the minimum relation, that matching frames to all frames must
> > > > > > > > > have.
> > > > > > > > > +The option value must be a double value between 0 and 1. The
> > > > > > > > > default value is 0.5. + at end table
> > > > > > > > > +
> > > > > > > > > + at subsection Examples
> > > > > > > > > +
> > > > > > > > > + at itemize
> > > > > > > > > + at item
> > > > > > > > > +To calculate the signature of an input video and store it in
> > > > > > > > > signature.xml: + at example
> > > > > > > > > +ffmpeg -i input.mkv -vf signature=filename=signature.xml -map 0:v
> > > > > > > > > -c rawvideo -f null - + at end example
> > > > > > > > 
> > > > > > > > the output seems to differ between 32 an 64bit x86
> > > > > > > > this would make any regression testing rather difficult
> > > > > > > > why is there a difference ? can this be avoided or would that result
> > > > > > > > in
> > > > > > > > some disadvantage ?
> > > > > > > 
> > > > > > > This is due to this line:
> > > > > > > sum -= ((double) blocksum)/(blocksize * denum);
> > > > > > > 
> > > > > > > sum was a double. It seems the difference leads to different results
> > > > > > > in 32 and 64 bit (the 5 decimal place). I have reworked the filter
> > > > > > > part so it does not use double at all. This also leads in some fewer
> > > > > > > divisions, but the numbers get really big. The relevant parts use
> > > > > > > int63_t.
> > > > > > > 
> > > > > > > If the videos gets really big, the numbers could overflow. Can I
> > > > > > > restrict this someway?
> > > > > > > 
> > > > > > > An upper bound could be find with:
> > > > > > > 255 * BLOCK_LCM * (width/32+1)^2 * (height/32+1)^2 < 2^63
> > > > > > > I tested it with 4K (UHD) input. This does not give any problems, but
> > > > > > > it is near the limit. (As a note: Especially 4K is a certain amount
> > > > > > > under the limit, because the width 3840 is dividable by 32, so the
> > > > > > > square in the above formula could be deleted)
> > > > > > > 
> > > > > > > The filter should generate the same signatures as in 64 bit before,
> > > > > > > now with 32 and 64 bit.> > 
> > > > > > if you really need more tha 64bit ints you can take a look at
> > > > > > libavutil/integer.h
> > > > > > it would be better if the operations can be reshuffled to keep using
> > > > > > intXY_t
> > > > > 
> > > > > This depends, IMHO 4K UHD is enough for now, and given, that you can
> > > > > simply rescale a higher resolution to somewhat below, without changing
> > > > > the function of the signature, I would simply add a check in config_input
> > > > > or so, that throws an error, if the resolution is too high. Would this be
> > > > > ok?
> > > > not optimal but ok i guess
> > > Because of this don't essentially change to function of the filter, what do you
> > > thing about this approach?:
> > > Check in config_input for a potentially overflow. If so, display a warning but
> > > don't quit. Instead divide all by 16 (or another meaningful power of 2).
> > > So you loose precision, but the filter keeps working and this won't affect the
> > > signature that much.
> > 
> > ok
> Implemented. I tested this with 5000x5000 and it works. If I calculated it right,
> the filter now supports pictures up to 551 billion pixels (an 740000x740000
> square). I also tested with a video below 4k in both modes. There are differences
> (ternary flips), but only very few (as expected).
> 
> Also fixed your previous comments and rebased to actual master.
> 
> Diff to old patch (last one I send) and the new one attached.
> 
> Gerion


this sefaults:

ffmpeg -i matrixbench_mpeg2.mpg -vf signature=xml:filename=signature.xml -f null -

[...]

-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety -- Benjamin Franklin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20160416/a477646b/attachment.sig>


More information about the ffmpeg-devel mailing list