[FFmpeg-devel] [PATCH 5/5] libavcodec/jpeg2000dec.c: Remove log2_chroma check in pixel format selection

[Michael Niedermayer]

On Mon, Jun 22, 2020 at 12:12:08AM +0530, gautamramk at gmail.com wrote:
> From: Gautam Ramakrishnan <gautamramk at gmail.com>
> 
> The log2_chroma_wh is derived from the sample separations of the
> codestream if the file is a j2k codestream. Not sure if sample
> separation is same is subsampling and whether using sample
> separation values from the codestream to determine pixel format.
> ---
>  libavcodec/jpeg2000dec.c | 2 --
>  1 file changed, 2 deletions(-)
> 
> diff --git a/libavcodec/jpeg2000dec.c b/libavcodec/jpeg2000dec.c
> index c8c89803ac..2b9659bf96 100644
> --- a/libavcodec/jpeg2000dec.c
> +++ b/libavcodec/jpeg2000dec.c
> @@ -225,8 +225,6 @@ static int pix_fmt_match(enum AVPixelFormat pix_fmt, int components,
>  
>      case 1:
>          match = match && desc->comp[0].depth >= bpc &&
> -                         (log2_chroma_wh >>  2 & 3) == 0 &&
> -                         (log2_chroma_wh       & 3) == 0 &&
>                           (desc->flags & AV_PIX_FMT_FLAG_PAL) == pal8 * AV_PIX_FMT_FLAG_PAL;
>      }
>      return match;

Heres a better bug report

ffmpeg_g -i afl-testcase/p0_10.j2k -f null -
(the testcase should be the normal reference file before fuzzing, this crashed
 before any fuzzing)

[...]
[jpeg2000 @ 0x555559d70880] bpno became invalid
    Last message repeated 4 times
Program received signal SIGSEGV, Segmentation fault.
ff_jpeg2000_cleanup (comp=0x555559d76d40, codsty=0x555559d7f98c) at libavcodec/jpeg2000.c:604
604	                    int nb_code_blocks = prec->nb_codeblocks_height * prec->nb_codeblocks_width;
(gdb) bt
#0  0x0000555557043463 in ff_jpeg2000_cleanup (comp=0x555559d76d40, codsty=0x555559d7f98c) at libavcodec/jpeg2000.c:604
#1  0x0000555557045bc5 in jpeg2000_dec_cleanup (s=s at entry=0x555559d71fc0) at libavcodec/jpeg2000dec.c:2029
#2  0x000055555706263c in jpeg2000_decode_frame (avctx=0x555559d70880, data=0x555559d71880, got_frame=0x7fffffffd5e0, avpkt=<optimized out>) at libavcodec/jpeg2000dec.c:2416
#3  0x0000555556aad261 in decode_simple_internal (frame=<optimized out>, avctx=<optimized out>) at libavcodec/decode.c:342
#4  0x0000555556aad261 in decode_simple_receive_frame (frame=<optimized out>, avctx=<optimized out>) at libavcodec/decode.c:538
#5  0x0000555556aad261 in decode_receive_frame_internal (avctx=avctx at entry=0x555559d70880, frame=0x555559d71880) at libavcodec/decode.c:556
#6  0x0000555556aafa28 in avcodec_send_packet (avctx=avctx at entry=0x555559d70880, avpkt=avpkt at entry=0x7fffffffd6b0) at libavcodec/decode.c:614
#7  0x00005555567c9f23 in try_decode_frame (s=s at entry=0x555559d6e940, st=st at entry=0x555559d6fe00, avpkt=avpkt at entry=0x555559d76940, options=<optimized out>) at libavformat/utils.c:3111
#8  0x00005555567fad2a in avformat_find_stream_info (ic=0x555559d6e940, options=0x555559d70740) at libavformat/utils.c:3954
#9  0x00005555558c6eb9 in open_input_file (o=o at entry=0x7fffffffdcb0, filename=<optimized out>) at fftools/ffmpeg_opt.c:1185
#10 0x00005555558d1ea9 in open_files (l=0x555559d6e718, l=0x555559d6e718, open_file=0x5555558c2410 <open_input_file>, inout=0x555558c178b9 "input") at fftools/ffmpeg_opt.c:3302
#11 0x00005555558d1ea9 in ffmpeg_parse_options (argc=<optimized out>, argv=<optimized out>) at fftools/ffmpeg_opt.c:3342
#12 0x00005555558abcb9 in main (argc=6, argv=0x7fffffffe258) at fftools/ffmpeg.c:4848

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

I have often repented speaking, but never of holding my tongue.
-- Xenocrates
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 195 bytes
Desc: not available
URL: <https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20200627/9839de33/attachment.sig>