[FFmpeg-devel] [RFC] Regression test speed

[Stefano Sabatini]

On date Saturday 2009-11-28 22:14:49 +0100, Michael Niedermayer encoded:
> Hi
> 
> On my duron make test eats over 17minutes. This is quite annoying as
> it limits which system i can reasonably work on.
> So id like to suggest that tests do not encode more tha 352x288x25
> pixels, that is codecs limited to high resolutions should use proportionally
> fewer frames.

Yes, especially the dnxhd tests are painfully slow.
 
> Also it seems the regression scripts have become quite convoluted and
> complex while still not even supporting running a subset of the tests.
> Or did i miss one?
> [iam not counting hacking the Makefile & reference files that of course
>  works]

Well it sometimes happen to me that I want to run only some tests, for
example I did a change which breaks one test, then I edit again the
code and I want to check only the failing test, this is not currently
possible without to hack Makefile & reference file.

This may be achievable putting the reference result of each single
test in a separate file, e.g.:

tests/rotozoom/test1.regression.ref
tests/rotozoom/test2.regression.ref
...
tests/lavf/test1.regression.ref
tests/lavf/test2.regression.ref
...

Or we could even create a separate dir per each test class:
tests/rotozoom/regression.sh
tests/rotozoom/regression.tests
tests/rotozoom/test1.ref
tests/rotozoom/test2.ref
...

So testing a given subset of tests would be as easy as editing the
regression.tests file.

Regards.
-- 
FFmpeg = Fierce and Fierce Mythic Patchable Elected Gadget