[FFmpeg-devel] avcodec/prores_ks : do not call estimate_alpha at each quantification step

[Martin Vignali]

>
> probably ok, check that this does not break alpha encoding.
>
>
Alpha encoding still work.

In fact this patch improve quality, when alpha encoding is used. (Not
intented at the start).
The modification came from the suppression of error arg and the move of
estimate_alpha.
Before this patch, estimate alpha, reset error to zero after slice
estimation.
After this patch, the error value of YUV slice estim is used.

Can be see using file
https://trac.ffmpeg.org/attachment/ticket/5995/source.png

./ffmpeg -i source.png -c:v prores_ks res.mov

After this patch, there is less blocking effect on edge

I will wait few days, if someone have other comments, before pushing.

Martin