[FFmpeg-devel] [PATCH v3] checkasm: add sample argument to adjust during bench

[Lynne]

On 21/05/2024 15:51, J. Dekker wrote:
> Some timers on certain device and test combinations can produce noisy
> results, affecting the reliability of performance measurements. One
> notable example of this is the Canaan K230 RISC-V development board.
> 
> An option to adjust the number of samples by an exponent (--runs) has
> been added, allowing developers to increase the sample count for more
> reliable results.
> 
> Signed-off-by: J. Dekker <jdek at itanimul.li>
> ---
>   tests/checkasm/checkasm.c | 16 +++++++++++++++-
>   tests/checkasm/checkasm.h |  7 ++++---
>   2 files changed, 19 insertions(+), 4 deletions(-)
> 
> diff --git a/tests/checkasm/checkasm.c b/tests/checkasm/checkasm.c
> index 31ca9f6e2b..669f2be9c1 100644
> --- a/tests/checkasm/checkasm.c
> +++ b/tests/checkasm/checkasm.c
> @@ -72,6 +72,9 @@
>   void (*checkasm_checked_call)(void *func, int dummy, ...) = checkasm_checked_call_novfp;
>   #endif
>   
> +/* Trade-off between speed and accuracy */
> +uint64_t bench_runs = 1U << 10;
> +
>   /* List of tests to invoke */
>   static const struct {
>       const char *name;
> @@ -820,7 +823,7 @@ static void bench_uninit(void)
>   static int usage(const char *path)
>   {
>       fprintf(stderr,
> -            "Usage: %s [--bench] [--test=<pattern>] [--verbose] [seed]\n",
> +            "Usage: %s [--bench] [--runs=<ptwo>] [--test=<pattern>] [--verbose] [seed]\n",
>               path);
>       return 1;
>   }
> @@ -867,6 +870,17 @@ int main(int argc, char *argv[])
>               state.test_name = arg + 7;
>           } else if (!strcmp(arg, "--verbose") || !strcmp(arg, "-v")) {
>               state.verbose = 1;
> +        } else if (!strncmp(arg, "--runs=", 7)) {
> +            l = strtoul(arg + 7, &end, 10);
> +            if (*end == '\0') {
> +                if (l > 30) {
> +                    fprintf(stderr, "checkasm: error: runs exponent must be within the range 0 <= 30\n");
> +                    usage(argv[0]);
> +                }
> +                bench_runs = 1U << l;
> +            } else {
> +                return usage(argv[0]);
> +            }
>           } else if ((l = strtoul(arg, &end, 10)) <= UINT_MAX &&
>                      *end == '\0') {
>               seed = l;
> diff --git a/tests/checkasm/checkasm.h b/tests/checkasm/checkasm.h
> index 07fcc751ff..e05053cbf6 100644
> --- a/tests/checkasm/checkasm.h
> +++ b/tests/checkasm/checkasm.h
> @@ -167,7 +167,7 @@ extern AVLFG checkasm_lfg;
>   
>   static av_unused void *func_ref, *func_new;
>   
> -#define BENCH_RUNS 1000 /* Trade-off between accuracy and speed */
> +extern uint64_t bench_runs;
>   
>   /* Decide whether or not the specified function needs to be tested */
>   #define check_func(func, ...) (checkasm_save_context(), func_ref = checkasm_check_func((func_new = func), __VA_ARGS__))
> @@ -336,10 +336,11 @@ typedef struct CheckasmPerf {
>               av_unused const int sysfd = perf->sysfd;\
>               func_type *tfunc = func_new;\
>               uint64_t tsum = 0;\
> -            int ti, tcount = 0;\
> +            uint64_t ti, tcount = 0;\
>               uint64_t t = 0; \
> +            const uint64_t truns = bench_runs;\
>               checkasm_set_signal_handler_state(1);\
> -            for (ti = 0; ti < BENCH_RUNS; ti++) {\
> +            for (ti = 0; ti < truns; ti++) {\
>                   PERF_START(t);\
>                   tfunc(__VA_ARGS__);\
>                   tfunc(__VA_ARGS__);\

Tested, works as intended
LGTM, thanks
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_0xA2FEA5F03F034464.asc
Type: application/pgp-keys
Size: 624 bytes
Desc: OpenPGP public key
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240521/68181def/attachment.key>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature.asc
Type: application/pgp-signature
Size: 236 bytes
Desc: OpenPGP digital signature
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20240521/68181def/attachment.sig>