[FFmpeg-devel] [PATCH v1 03/13] lavc/hevc_ps: Add SPS/PPS parse support for HEVC extension syntax

[James Almer]

On 12/1/2022 5:13 AM, Fei Wang wrote:
> +static void colour_mapping_octants(GetBitContext *gb, HEVCPPS *pps, int inp_depth,
> +                                   int idx_y, int idx_cb, int idx_cr, int inp_length)
> +{
> +    uint8_t split_octant_flag, coded_res_flag;
> +    uint16_t part_num_y, res_coeff_q, res_coeff_r;

Use int or unsigned for scalar values in stack.

> +    int bit_depth_cm_input_y, bit_depth_cm_output_y, cm_res_bits;
> +    int k, m, n, c, i, j;

You can reduce the scope of almost all the above variables.

> +
> +    part_num_y = 1 << pps->cm_y_part_num_log2;
> +
> +    if (inp_depth < pps->cm_octant_depth)
> +        split_octant_flag = get_bits1(gb);
> +
> +    if (split_octant_flag)

split_octant_flag may be undefined here. It should be initialized to 0.
This is probably the source of the issue Michael reported.

> +        for (k = 0; k < 2; k++)

for (int k = 0...)

Same for the rest.

> +            for (m = 0; m < 2; m++)
> +                for (n = 0; n < 2; n++)
> +                    colour_mapping_octants(gb, pps, inp_depth + 1,
> +                                           idx_y + part_num_y * k * inp_length / 2,
> +                                           idx_cb + m * inp_length / 2,
> +                                           idx_cr + n * inp_length / 2,
> +                                           inp_length / 2);
> +    else
> +        for (i = 0; i < part_num_y; i++) {
> +            for (j = 0; j < 4; j++) {
> +                coded_res_flag = get_bits1(gb);
> +                if (coded_res_flag)
> +                    for (c = 0; c < 3; c++) {
> +                        res_coeff_q = get_ue_golomb_long(gb);
> +                        bit_depth_cm_input_y = 8 + pps->luma_bit_depth_cm_input_minus8;
> +                        bit_depth_cm_output_y = 8 + pps->luma_bit_depth_cm_output_minus8;
> +                        cm_res_bits = FFMAX(0, 10 + bit_depth_cm_input_y - bit_depth_cm_output_y -
> +                                            pps->cm_res_quant_bits - (pps->cm_delta_flc_bits_minus1 + 1));
> +                        res_coeff_r = get_bits(gb, cm_res_bits);
> +                        if (res_coeff_q || res_coeff_r)
> +                            skip_bits1(gb);
> +                    }
> +            }
> +        }
> +}