[FFmpeg-devel] Adding FATE tests for FFV1 - revisited
t.rapp at noa-archive.com
Mon Aug 31 13:44:08 CEST 2015
On 30.08.2015 21:32, Michael Niedermayer wrote:
> On Sun, Aug 30, 2015 at 07:06:44PM +0200, Peter B. wrote:
>> I've been working on FATE tests for FFV1 in the past already . My
>> tests didn't work on all platforms and therefore never made it upstream.
>> I think it's better if I try to provide these new tests in smaller
>> chunks now :)
>> First of all, there are things I find inconsistent or confusing with the
>> current tests (vcodec.mak):
>> - ENCOPTS for FFV1.3 contain "-vcodec ffv1" instead of "CODEC=ffv1"
>> (this generates "-c ffv1.3" as parameter?)
>> - Target "fate-vsynth%-*" tests default to sws_flags
>> "accurate_rnd+bitexact". FFV1.3 tests have "neighbor+bitexact". Why?
> it makes more cases lossless IIRC
> the default upscaling + default downscaling is not binary identical
>> - ENCOPTS for "fate-vsynth%-ffv1" are "-slices 4", which is an
>> FFV1.3-only option.
I assume the FFV1 level/subversion is auto-negotiated if some explicit
"-level" option is missing (e.g. see libavcodec/ffv1enc.c line 680).
>> - What is "ffv1.0"?
>> My ideas/plans would be something like this:
>> First steps:
>> 1) Clean the current FFV1 tests (naming, ENCDEC options, etc)
>> 2) Move FFV1 tests to its own file (ffv1.mak). Or at least to
>> 3) Have separate tests for different FFV1 versions (1,3)
>> 4) Add default argument "-g 1"
This removes the test for the default GOP size.
>> 5) Add tests to cover the following cases:
>> - Color spaces YUV, RGB, GRAY
>> - bits-per-component as currently supported
>> - YUV subsampling 420, 422, 444
>> - Alpha channel: YUVA, BGRA
>> 6) Multiple coder/context options
>> 7) Multiple slices options
>> 8) Testing SliceCRC
>> This will produce quite a number of tests :(
>> I guess it is desired to keep the number of tests as low as necessary?
> avoiding redundant tests would be a good idea
>> I've attached my old test Makefile (ffv1.mak), for reference.
>> What is the best way to proceed?
> probably, send patches
> and probably better few and small ones at once then wait and see
> in which direction reviewes go before spending too much time in some
> specific direction
I agree that having a full test suite for FFV1 would be a nice thing but
am not sure if integration into FATE is the right framework for it. As
far as I understand FATE tests need a good balance between processing
time and code coverage.
Maybe it can be part of the IETF standardization task to create a FFV1
test environment which tests every color-space/bit-depth/subsampling/...
Just my thoughts,
More information about the ffmpeg-devel