[FFmpeg-devel] [PATCH v4 2/2][GSoC 2024] tests/checkasm: Add check_vvc_sad to vvc_mc.c

[Martin Storsjö]

On Tue, 21 May 2024, Rémi Denis-Courmont wrote:

> Hi,
>
> Le 20 mai 2024 03:42:03 GMT+03:00, Stone Chen <chen.stonechen at gmail.com> a écrit :
>> Adds checkasm for DMVR SAD AVX2 implementation.
>>
>> Benchmarks ( AMD 7940HS )
>> vvc_sad_8x8_c: 70.0
>> vvc_sad_8x8_avx2: 10.0
>> vvc_sad_16x16_c: 280.0
>> vvc_sad_16x16_avx2: 20.0
>> vvc_sad_32x32_c: 1020.0
>> vvc_sad_32x32_avx2: 70.0
>> vvc_sad_64x64_c: 3560.0
>> vvc_sad_64x64_avx2: 270.0
>> vvc_sad_128x128_c: 13760.0
>> vvc_sad_128x128_avx2: 1070.0
>> ---
>> tests/checkasm/vvc_mc.c | 38 ++++++++++++++++++++++++++++++++++++++
>> 1 file changed, 38 insertions(+)
>
> VVC benchmarks have increased checksam runtime by at least an order of 
> magnitude. It's become so prohibitively slow that I could not even get 
> to the end.
>
> This is not an acceptable situation and impedes non-VVC assembler work

I don't quite understand; whenever benchmarking anything in checkasm, I 
would always run e.g. "checkasm --test=ac3dsp 
--bench=ac3_sum_square_bufferfly_float", limiting the total running of 
tests to a specific module, and only benchmarking a subset of the run 
functions. (The --bench parameter specifies a prefix; only functions 
matching that prefix gets benchmarked.)

Without limiting the scope with a --test parameter, checkasm benchmarking 
has always been prohibitively slow for me - so I don't think there's 
anything new here? If you were lucky enough to be able to do a full run of 
checkasm with benchmarks of all functions before, that sounds like an 
exception to me, not a reason to limit adding new tests?

That said I'm not familiar with the VVC tests in checkasm, perhaps they 
benchmark things excessively. But I don't see how that would impede work 
on other DSP functions in any way?

// Martin