[FFmpeg-devel] [RFC]lswr/rematrix: Support s32p

[Muhammad Faiz]

On Sun, Oct 29, 2017 at 3:55 AM, Carl Eugen Hoyos <ceffmpeg at gmail.com> wrote:
> Hi!
>
> Attached patch fixes a random testcase for ticket #6785 here but I
> don't know if this is the correct fix.
>
> Please review, Carl Eugen
>
> From a93b9309d74f5eadece371ee1e682d266af6cd83 Mon Sep 17 00:00:00 2001
> From: Carl Eugen Hoyos <ceffmpeg at gmail.com>
> Date: Sat, 28 Oct 2017 22:52:02 +0200
> Subject: [PATCH] lswr/rematrix: Support s32p.
>
> Fixes ticket #6785.
> ---
>  libswresample/rematrix.c |   21 +++++++++++++++------
>  1 file changed, 15 insertions(+), 6 deletions(-)
>
> diff --git a/libswresample/rematrix.c b/libswresample/rematrix.c
> index 66a43c1..a6aa6b0 100644
> --- a/libswresample/rematrix.c
> +++ b/libswresample/rematrix.c
> @@ -445,14 +445,23 @@ av_cold int swri_rematrix_init(SwrContext *s){
>          s->mix_2_1_f = (mix_2_1_func_type*)sum2_double;
>          s->mix_any_f = (mix_any_func_type*)get_mix_any_func_double(s);
>      }else if(s->midbuf.fmt == AV_SAMPLE_FMT_S32P){
> -        // Only for dithering currently
> -//         s->native_matrix = av_calloc(nb_in * nb_out, sizeof(double));
> -        s->native_one    = av_mallocz(sizeof(int));
> +        s->native_one    = av_mallocz(sizeof(int64_t));
>          if (!s->native_one)
>              return AVERROR(ENOMEM);
> -//         for (i = 0; i < nb_out; i++)
> -//             for (j = 0; j < nb_in; j++)
> -//                 ((double*)s->native_matrix)[i * nb_in + j] = s->matrix[i][j];
> +        s->native_matrix = av_calloc(nb_in * nb_out, sizeof(int));
> +        if (!s->native_matrix) {
> +            av_freep(&s->native_one);
> +            return AVERROR(ENOMEM);
> +        }
> +        for (i = 0; i < nb_out; i++) {
> +            double rem = 0;
> +
> +            for (j = 0; j < nb_in; j++) {
> +                double target = s->matrix[i][j] * 32768 + rem;
> +                ((int*)s->native_matrix)[i * nb_in + j] = lrintf(target);
> +                rem += target - ((int64_t*)s->native_matrix)[i * nb_in + j];
> +            }
> +        }
>          *((int*)s->native_one) = 32768;
>          s->mix_1_1_f = (mix_1_1_func_type*)copy_s32;
>          s->mix_2_1_f = (mix_2_1_func_type*)sum2_s32;

The code is confusing.
Which is the type of native_matrix and native_one? int or int64_t?