Henrik Gramner henrik at gramner.com
Thu Sep 24 20:41:28 CEST 2015

```On Tue, Sep 22, 2015 at 7:26 PM, Ronald S. Bultje <rsbultje at gmail.com> wrote:
> +    for (k = 0; k < sz; k++) {
> +        out[k] = 0.0;
> +        for (n = 0; n < sz; n++)
> +            out[k] += in[n] * cos(M_PI * (2 * n + 1) * k / (sz * 2.0));
> +        if (k == 0)
> +            out[k] *= M_SQRT1_2;
> +    }

You could move the last row outside the for loop.

> +// see "Towards jointly optimal spatial prediction and adaptive transform in
> +// video/image coding", by J. Han, A. Saxena, and K. Rose
> +// IEEE Proc. ICASSP, pp. 726-729, Mar. 2010.
> +static void fadst4_1d(double *out, const double *in, int sz)
[...]
> +// see "A Butterfly Structured Design of The Hybrid Transform Coding Scheme",
> +// by Jingning Han, Yaowu Xu, and Debargha Mukherjee

Thumbs up for giving sources.

+static void ftx(int16_t *out, const int16_t *in, enum TxfmMode tx,
+                enum TxfmType txtp, int sz, int bit_depth)

This function is only ever called with in == out, so you could merge
those parameters.

+                            memcpy(subcoef0, coef, sz * sz * 2 * SIZEOF_PIXEL);
[...]
+                        memcpy(subcoef1, subcoef0, sz * sz * 2 * SIZEOF_PIXEL);

I'd define a SIZEOF_COEF or similar for this instead of using
SIZEOF_PIXEL on something that's aren't pixels.

I didn't feel like reading through two research papers so I won't
comment on your calculations for creating input coefficients, but the
testing code seems ok.
```