[FFmpeg-devel] [PATCH 1/2] test: hlsenc: Don't generate test data with -re

[James Almer]

On 8/7/2020 5:35 PM, Michael Niedermayer wrote:
> On Thu, Aug 06, 2020 at 10:08:33PM +0300, Martin Storsjö wrote:
>> On Mon, 3 Aug 2020, Martin Storsjö wrote:
>>
>>> On Mon, 3 Aug 2020, Michael Niedermayer wrote:
>>>
>>>> is this removing the only test which tests the "-re" flag ?
>>>> if so that would reduce test coverage
>>>
>>> Well, tehcnically yes, but this test doesn't actually test whether the
>>> "-re" flag does what it's supposed to or anything like that - it only
>>> tests that adding the flag doesn't error out and doesn't alter the
>>> output. 
> 
> yes
> 
> 
>>> And it looks very much unintentional here.
> 
> maybe
> 
> 
>>
>> Do you want to follow up the discussion here? I'd like at least some sort of
>> acknowledgement back on the discussion before going forward with it in any
>> direction.
> 
> sorry for the lack of reply, theres always more to reply to and do then i seem
> to have time.
> 
> About it not fully testing things, that is of course not great but testing
> things halfway is still better than not testing at all.
> 
> Also it may make sense given the large number of features we have to test
> multiple per test.
> For example with 200 features, it would take 200 individual tests to test each
> in its own test, but only 10 tests if 4 of these would always test the same 
> feature. this of course assumes all kinds of things, its more meant to show
> that a bit of "chaos" in what each test tests could actually improve the
> amount of information we could obtain from the pattern of failing tests
> 
> Thanks

If it's just about not removing coverage/testing of -re, it could be
added to some other test with less frames than this one, so it doesn't
take five seconds.