[FFmpeg-devel] [PATCHv3] af_hdcd: more FATE tests

[Burt P.]

On Sun, Dec 18, 2016 at 6:54 PM, Michael Niedermayer
<michael at niedermayer.cc> wrote:
> On Sun, Dec 18, 2016 at 12:48:45PM -0600, Burt P wrote:
>> Additional/Modified FATE tests improve code coverage from 63.7% to 98.1%.
>>
>> Changed fate-suite sample files:
>> * filter/hdcd-encoding-errors.flac (1.3M) replaced by
>>   a smaller version (140K). It can be replaced because the test
>>   only looks for a non-zero number of errors, so the existing test
>>   will still pass.
>
> IMO replacing files is not ok
> it would change all past instances of the related fate test
> a bug report refering to a fate failure could become unreproduceable
> or otherwise working fate tests could start failing ...
> as much as i prefer to safe a few bytes in this case i prefer to waste
> some space over the potential problems
>
> you can add files, but not remove or replace unless they are truly
> unused by every past checkout

Well, I don't mind if it is kept, I was just trying to offset the additions.
I understand why removing or replacing samples is generally not a good
idea, but I would point out that in this case, the file could be replaced
without changing the result in any past revision of the test.
This particular test outcome is determined by grepping a line out of the
log that reports the number of errors encountered, and it only has to be
non-zero, so the shorter sample with fewer errors (but at least one) still
passes any old revision.

-- 
Burt