[FFmpeg-devel] [PATCH 3/4 v2] avcodec/vc1: rewrite vc1_decode_i_blocks to align with VC-1 spec

[Michael Niedermayer]

On Fri, Jun 08, 2018 at 11:01:30AM +0200, Jerome Borsboom wrote:
> Change vc1_decode_i_blocks to use vc1_put_blocks_clamped and
> ff_vc1_i_loop_filter.
> 
> Signed-off-by: Jerome Borsboom <jerome.borsboom at carpalis.nl>
> ---
>  libavcodec/vc1_block.c | 77 +++++++++++++++++++-------------------------------
>  1 file changed, 29 insertions(+), 48 deletions(-)

crashes:

[vc1 @ 0x11b77200] Slice header damaged
==2065==    at 0x120D69C: VALGRIND_PRINTF_BACKTRACE (valgrind.h:4550)
==2065==    by 0x120E11C: av_log_default_callback (log.c:351)
==2065==    by 0x120E2BB: av_vlog (log.c:377)
==2065==    by 0x120E27B: av_log (log.c:369)
==2065==    by 0xC3B5B5: vc1_decode_frame (vc1dec.c:1014)
==2065==    by 0x8A6BAB: decode_simple_internal (decode.c:398)
==2065==    by 0x8A7832: decode_simple_receive_frame (decode.c:594)
==2065==    by 0x8A78FD: decode_receive_frame_internal (decode.c:612)
==2065==    by 0x8A7B75: avcodec_send_packet (decode.c:674)
==2065==    by 0x43373E: decode (ffmpeg.c:2238)
==2065==    by 0x433F98: decode_video (ffmpeg.c:2382)
==2065==    by 0x434FE0: process_input_packet (ffmpeg.c:2623)
==2065==    by 0x43C207: process_input (ffmpeg.c:4461)
==2065==    by 0x43C7B0: transcode_step (ffmpeg.c:4581)
==2065==    by 0x43C92C: transcode (ffmpeg.c:4635)
==2065==    by 0x43D199: main (ffmpeg.c:4842)
==2065== Invalid read of size 8
==2065==    at 0xD790BD: ??? (libavcodec/x86/vc1dsp_loopfilter.asm:302)
==2065==    by 0xC26AA6: vc1_i_v_loop_filter (vc1_loopfilter.c:239)
==2065==    by 0xC26BFB: ff_vc1_i_loop_filter (vc1_loopfilter.c:266)
==2065==    by 0xC23C72: vc1_decode_i_blocks (vc1_block.c:2594)
==2065==    by 0xC259D4: ff_vc1_decode_blocks (vc1_block.c:2969)
==2065==    by 0xC3B7D6: vc1_decode_frame (vc1dec.c:1042)
==2065==    by 0x8A6BAB: decode_simple_internal (decode.c:398)
==2065==    by 0x8A7832: decode_simple_receive_frame (decode.c:594)
==2065==    by 0x8A78FD: decode_receive_frame_internal (decode.c:612)
==2065==    by 0x8A7B75: avcodec_send_packet (decode.c:674)
==2065==    by 0x43373E: decode (ffmpeg.c:2238)
==2065==    by 0x433F98: decode_video (ffmpeg.c:2382)
==2065==    by 0x434FE0: process_input_packet (ffmpeg.c:2623)
==2065==    by 0x43C207: process_input (ffmpeg.c:4461)
==2065==    by 0x43C7B0: transcode_step (ffmpeg.c:4581)
==2065==    by 0x43C92C: transcode (ffmpeg.c:4635)
==2065==    by 0x43D199: main (ffmpeg.c:4842)
==2065==  Address 0x11f25100 is not stack'd, malloc'd or (recently) free'd

....

==2065== Process terminating with default action of signal 11 (SIGSEGV)
==2065==  General Protection Fault
==2065==    at 0x13786A6: ??? (in ffmpeg/ffmpeg_g)
==2065==    by 0x11F8D78: av_buffer_pool_uninit (buffer.c:285)
==2065==    by 0xBF5F7B: avcodec_close (utils.c:1089)
==2065==    by 0x43CCB5: transcode (ffmpeg.c:4697)
==2065==    by 0x43D199: main (ffmpeg.c:4842)

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

No snowflake in an avalanche ever feels responsible. -- Voltaire
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20180609/c9fdadc2/attachment.sig>