[FFmpeg-devel] [RFC] Lavfi test system
Sun Dec 13 03:33:18 CET 2009
On Sun, Dec 13, 2009 at 01:52:41AM +0100, Stefano Sabatini wrote:
> On date Sunday 2009-11-29 13:15:17 +0100, Michael Niedermayer encoded:
> > On Sun, Nov 29, 2009 at 02:05:08AM +0100, Stefano Sabatini wrote:
> > [...]
> > > seems to depend on the slicification of the image (yes at this point
> > > is clear that we need some testing system for spotting all these
> > > problems).
> > it was clear since years that lavfi tests are required to have a
> > useable lavfi, its too complex and easy to break. But ive been
> > ignored and didnt had time to write it myself
> > it would be nice if some regression tests would be added before
> > further lavfi work is done.
> > The minimum filter chain to test must contain all filter pairs,
> > that makes ~ n^2 filters in it or n^2 tests for n distinct filters.
> If we have n^2 tests for each filter, then we end-up with n^3 tests,
n tests for each filter makes n^2 which is what i meant
> We already had had the experience of filters which only failed when
> used with a particular format and with a special value for slices
> height / input size, so I don't think that to design a system to test
> all the possible cases would be something achievable.
we should test a widely spread mix of things ...
> Also when we consider filters with a more/less than 1 input and 1
> output things get even more complicated (for example how to test when
> a sink fails?).
ask me when we have a sink
> A very simple alternative may be to simply implement for each filter a
> set of carefully handcrafted tests which replicate all the possible
> critical corner cases which may lead to a failure.
Iam not sure if this is an alternative. Iam not even sure if what i think
you talk about is what you talk about.
A test filter that would feed another filter with pictures with differnt
sets of permissions, sizes, pix_fmts, slice order and size, interlacing
and progressive, misaligned and aligned data, ....
is surely usefull, its also easy to test as its output should always be the
same (or very close for differing pix_fmts)
still we need actual tests of filter interactions of actual filter chains,
its not the same thing.
> > and then each filter should be individually tested against all
> > colorspaces.
> Not every pixel format will be in general supported, so we may
> manually add a list of supported formats. Even better would be to add
> some interface to expose the set of supported inputs (though not sure
> this is possible, especially considering the scale filter).
you are searching for query_formats() ?
> And BTW, in any case I believe need a more modular test system, it
> should be easy to simply edit some list and test just a subset of the
> complete test-set.
Michael GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB
Complexity theory is the science of finding the exact solution to an
approximation. Benchmarking OTOH is finding an approximation of the exact
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 189 bytes
Desc: Digital signature
More information about the ffmpeg-devel