[Ffmpeg-devel] few remarks for h264 decoder

[Gábor Kovács]

Hi,

I think I found another one, but it's mmx releated which I don't actually use and I don't
even have sample clip for this, so there is a chance I'am wrong.

9.
It's about deblocking. alpha_table[] has zero values for low qscale. In this case 
ABS( p0 - q0 ) < alpha should always fail. The problem with mmx code that it uses alpha-1. 
In H264_DEBLOCK_MASK() packuswb will saturate -1 to 0 and the |p0-q0| > alpha-1
condition may allow deblocking when |p0-q0| = 0. Probably rare problem.

I made a mistake in my last post about point 8. (when direct_8x8_inference_flag=1).
It's not only direct_spatial_mv_pred related=1 related. As far as I see 
direct_spatial_mv_pred related=0 can have the same problem (when having four different
mvs as reference[1][0] inside a direct 8x8 subblock).

bye, Picard