[FFmpeg-devel] [RFC] Lavfi test system

Stefano Sabatini stefano.sabatini-lala
Wed Mar 10 01:59:10 CET 2010


On date Monday 2010-03-08 23:02:10 +0100, Michael Niedermayer encoded:
> On Mon, Mar 08, 2010 at 10:40:02PM +0100, Stefano Sabatini wrote:
> > On date Monday 2010-03-08 03:17:11 +0100, Michael Niedermayer encoded:
> > > On Mon, Mar 08, 2010 at 02:36:14AM +0100, Stefano Sabatini wrote:
> > > > On date Sunday 2010-03-07 17:44:20 +0100, Michael Niedermayer encoded:
> > > > > On Sun, Mar 07, 2010 at 04:20:16PM +0100, Stefano Sabatini wrote:
> > > > > > On date Sunday 2010-03-07 11:18:33 +0100, Stefano Sabatini encoded:
> > > > > > > On date Thursday 2010-01-07 23:36:02 +0100, Stefano Sabatini encoded:
> > > > > [...]
> > > > > > Index: libavfilter-soc/ffmpeg/tests/lavfi-regression.sh
> > > > > > ===================================================================
> > > > > > --- libavfilter-soc.orig/ffmpeg/tests/lavfi-regression.sh	2010-03-07 16:15:22.000000000 +0100
> > > > > > +++ libavfilter-soc/ffmpeg/tests/lavfi-regression.sh	2010-03-07 16:16:43.000000000 +0100
> > > > > > @@ -20,6 +20,8 @@
> > > > > >      vfilters=$2
> > > > > >  
> > > > > >      if [ -n "$test" ] ; then
> > > > > > +        slicify_h=$(expr 8 + $(date +%N) % 24)
> > > > > > +        vfilters=slicify=$slicify_h,$vfilters
> > > > > >          do_video_encoding ${test_name}.nut "" "-vcodec rawvideo -vfilters $vfilters"
> > > > > >      fi
> > > > > >  }
> > > > > 
> > > > > this is a problem because its not reproduceable
> > > > > 
> > > > > an LCG started from a constant would be better
> > > > 
> > > > I believed the idea was to try different slicing at every run... what
> > > > am I missing?
> > > 
> > > we dont want spurious unreproduceable failures
> > > or other way around spurious unreproduceable successes in teh test wen
> > > they should fail
> > > it makes debuging hard ...
> > 
> > I agree but this way we are using only a limited subset of slice
> > values.
> 
> true, could we postpone resolving this though.
> having some regression tests is very important
> testing all slice sizes is not that important
> 
> 
> > 
> > Another solution would be to iterate a few times each test with
> > different slice height values taken from a pseudo random
> > sequence. Would this be OK?
> 
> that slows it down proportionally
> maybe we could add a "r" parameter to slicify to make it produce
> deterministic but random slice sizes that would test more than a
> single size for a video

Thinking more about it, that looked like the best solution, see the
attached patch (which depends on that one extending slicify which I just
posted in a new thread).

> > > > Regards.
> > > > -- 
> > > > FFmpeg = Frenzy and Fostering Miracolous Problematic Erroneous Guru
> > > 
> > > >  lavfi-regression.sh |   10 ++++++++++
> > > >  1 file changed, 10 insertions(+)
> > > > b361118ffebf15ed75e0bac79f4aa27b4d20d371  randomize-lavfi-test-slice-sizes.patch
> > > > Index: libavfilter-soc/ffmpeg/tests/lavfi-regression.sh
> > > > ===================================================================
> > > > --- libavfilter-soc.orig/ffmpeg/tests/lavfi-regression.sh	2010-03-07 16:17:19.000000000 +0100
> > > > +++ libavfilter-soc/ffmpeg/tests/lavfi-regression.sh	2010-03-08 02:29:41.000000000 +0100
> > > > @@ -14,12 +14,22 @@
> > > >  rm -f "$logfile"
> > > >  rm -f "$benchfile"
> > > >  
> > > > +get_lcg()
> > > > +{
> > > > +    echo "65539 * $1 % (2^31-1)" | bc
> > > 
> > > where does this generator come from?
> > 
> > Uh an experiment of mine, I did some little tests on it and it looked
> > good enough for this application.
> 
> dont try to be smarter than scientists who spend years analyzing the
> properties of these generators.
> 
> 
> > 
> > > and why dont you use the generator we use elsewhere in ffmpeg?
> > 
> > Do you mean av_lfg_get()? Are you suggesting that I should use a C
> > program (lfg_get.c) for generating such a sequence?
> 
> use this:
> state= state*1664525+1013904223
> 
> and dont forget to use *24 >>32, do not use %24

I came out wit this:

lcg_get() { echo "($1 * 1664525 + 1013904223) % 2^32" | bc; }

bc does not know shift operators.

> > > also what should be used as random value is the scaled down value not the
> > > modulo something value like you do later.
> > 
> > I have to say that I lack the understanding of the theory behind,
> > please if you can give a very short explanation, I'm looking at the
> > code using av_lfg_get() and most of the times a modulo operation is
> > performed.
> 
> just use a proofen standard generator dont invent your own, someone
> knuth? said something like, random functions are poor random number genrators
> this one hits the key issue ...
> 
> If you want to design a better LCG, look at some prng test programs like
> the diehard tests or TestU01
> (note some of the soucre code [maybe debian package?] of diehard was in a
>  very sorry state you wont get that one working correctly if its still in
>  that condition)

Thanks for the explanation, I took a quickly study into PRNG, the one
you suggest seems the same adopted/proposed in Numerical Recipes.

Regards.
-- 
FFmpeg = Furious and Fascinating Mythic Power Enlightening Genius
-------------- next part --------------
A non-text attachment was scrubbed...
Name: randomize-lavfi-test-slice-sizes.patch
Type: text/x-diff
Size: 567 bytes
Desc: not available
URL: <http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20100310/fc520283/attachment.patch>



More information about the ffmpeg-devel mailing list