[FFmpeg-devel] [PATCH] fix vp6-huffman sort order

[Måns Rullgård]

pross at xvid.org writes:

> Hi.
>
> This patch corrects an assumption made re qsort ordering when decoding
> VP6-huffman files.
>
> The current code works by-accident with GLIC, but fails when using
> Microsoft qsort implementation (and presumably others).
>
> Cheers,
>
> -- Peter
> (A907 E02F A6E5 0CD2 34CD 20D2 6760 79C5 AC40 DD6B)
>
> Index: libavcodec/vp6.c
> ===================================================================
> --- libavcodec/vp6.c	(revision 13869)
> +++ libavcodec/vp6.c	(working copy)
> @@ -202,10 +202,11 @@
>                  model->vector_fdv[comp][node] = vp56_rac_gets_nn(c, 7);
>  }
>  
> +/* sort comparator: nodes must ascend by count, but with descending symbol order */
>  static int vp6_huff_cmp(const void *va, const void *vb)
>  {
>      const Node *a = va, *b = vb;
> -    return a->count >= b->count;
> +    return (a->count - b->count)*12 + (b->sym - a->sym);
>  }

*16 would be faster.  What is the range of ->count?

-- 
M?ns Rullg?rd
mans at mansr.com