[FFmpeg-devel] [PATCHv2] fate: Add an option for disabling the 2k/4k tests

[Carl Eugen Hoyos]

> Am 13.12.2019 um 09:34 schrieb Martin Storsjö <martin at martin.st>:
> 
>> On Thu, 12 Dec 2019, James Almer wrote:
>> 
>>> On 12/12/2019 7:03 PM, Martin Storsjö wrote:
>>>> On Wed, 11 Dec 2019, Martin Storsjö wrote:
>>>>> On Wed, 11 Dec 2019, Carl Eugen Hoyos wrote:
>>>>> 
>>>>> Am Mi., 11. Dez. 2019 um 09:39 Uhr schrieb Martin Storsjö 
>>>> <martin at martin.st>:
>>>>>> 
>>>>>> When testing on a memory limited system, these tests consume a
>>>>>> significant amount of memory and can often fail if testing by running
>>>>>> multiple processes in parallel.
>>>>>> ---
>>>>>> Adjusted to use ALLYES instead of a -yes-yes construct.
>>>>>> 
>>>>>> Also moved the 2k tests to the same option.
>>>>>> ---
>>>>>>  configure             | 3 +++
>>>>>>  tests/fate/seek.mak   | 3 ++-
>>>>>>  tests/fate/vcodec.mak | 5 +++--
>>>>>>  3 files changed, 8 insertions(+), 3 deletions(-)
>>>>>> 
>>>>>> diff --git a/configure b/configure
>>>>>> index ca7137f341..922cd8d0ee 100755
>>>>>> --- a/configure
>>>>>> +++ b/configure
>>>>>> @@ -482,6 +482,7 @@ Developer options (useful when working on FFmpeg 
>>>> itself):
>>>>>>    --ignore-tests=TESTS     comma-separated list (without "fate-"
>>>>>> prefix
>>>>>>                             in the name) of tests whose result is
>>>>>> ignored
>>>>>>    --enable-linux-perf      enable Linux Performance Monitor API
>>>>>> +  --disable-large-tests    disable tests that use a large amount of 
>>>> memory
>>>>> 
>>>>> I would have suggested to control this when running the tests, if the 
>>>> configure
>>>>> setting makes sense, it should at least be possible to change the
>>>>> setting 
>>>> when
>>>>> calling make.
>>>>> Or is that possible anyway?
>>>> 
>>>> It's possible to do e.g. "make fate CONFIG_LARGE_TESTS=no"; any
>>>> var=value assignment on the make command line overrides any
>>>> var=othervalue assignment within the makefiles themselves, but that
>>>> doesn't seem very convenient.
>>>> 
>>>> But I'd like to have it as a configure option, to easily be able to
>>>> set it e.g. in a fate setup.
>>> Any further opinions on this one - is it ok to go ahead with it in this
>>> form, or are changes requested?
>> 
>> Configure option is fine if it can also be overridden from the command
>> line at runtime (like --samples and SAMPLES).
> 
> Well, it can be overridden at runtime, but it's not with a very convenient name (the CONFIG_* variable). Is that ok?

Ideally, this would be possible with:
make BIG=no fate

Carl Eugen