[FFmpeg-devel] [PATCH v2] fate: skip encoded output hashing for tests that depend on zlib

[Martin Storsjö]

On Sat, 19 Oct 2024, James Almer wrote:

> On 10/19/2024 11:04 AM, Martin Storsjö wrote:
>> On Sat, 19 Oct 2024, James Almer wrote:
>> 
>>> As there's no guarantee that the output will be exactly the same 
>>> across implementations,
>>> just don't test it at all.
>>> 
>>> Signed-off-by: James Almer <jamrial at gmail.com>
>>> ---
>>> All tests covered now. Do we go this route, or update the zlib version 
>>> of the
>>> failing FATE machines?
>> 
>> I would say that requiring an upgrade isn't reasonsble here.
>> 
>> This isn't only about fate.ffmpeg.org, but also about any developer - 
>> and user compiling ffmpeg for themselves - to be able to verify that 
>> their build is good.
>> 
>> If we go down the "enforce upgrades" path, we pretty much force all 
>> users to upgrade or stop using zlib. For still supported, not entirely 
>> ancient distributions like Ubuntu 20.04, that seems a bit harsh for 
>> something which is mainly an issue with our tests.
> But are those distros affected? Which zlib builds are faulty?

I don't remember exactly which versions that are affected (Ramiro pointed 
it out in some message), but at least Ubuntu 20.04 is affected.

// Martin