[FFmpeg-trac] #4301(avfilter:open): -vf scale=1280:720, hqdn3d=5:4:7:6 and -pix_fmt yuv420p10le cause crash
FFmpeg
trac at avcodec.org
Wed May 27 00:56:27 CEST 2015
#4301: -vf scale=1280:720,hqdn3d=5:4:7:6 and -pix_fmt yuv420p10le cause crash
-------------------------------------+-------------------------------------
Reporter: Selur | Owner:
Type: defect | Status: open
Priority: important | Component: avfilter
Version: git-master | Resolution:
Keywords: hqdn3d | Blocked By:
crash SIGSEGV regression | Reproduced by developer: 1
Blocking: |
Analyzed by developer: 0 |
-------------------------------------+-------------------------------------
Changes (by cehoyos):
* keywords: => hqdn3d crash SIGSEGV regression
* priority: normal => important
* status: new => open
* component: undetermined => avfilter
* reproduced: 0 => 1
Comment:
Could be considered a regression since 1ad715db
{{{
(gdb) r -i out.avi -vf hqdn3d=5:4:7:6 -f null -
Starting program: ffmpeg_g -i out.avi -vf hqdn3d=5:4:7:6 -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-72360-g816005d Copyright (c) 2000-2015 the FFmpeg
developers
built with gcc 4.7 (SUSE Linux)
configuration: --enable-gpl
libavutil 54. 25.100 / 54. 25.100
libavcodec 56. 41.100 / 56. 41.100
libavformat 56. 33.101 / 56. 33.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 16.101 / 5. 16.101
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
out.avi: No such file or directory
[Inferior 1 (process 14801) exited with code 01]
(gdb) r -i untitled.split.2_new.split.1.cut.avi -vf hqdn3d=5:4:7:6 -f null
-
Starting program: /home/cehoyos/test/cehoyos/FFmpeg/ffmpeg_g -i
untitled.split.2_new.split.1.cut.avi -vf hqdn3d=5:4:7:6 -f null -
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
ffmpeg version N-72360-g816005d Copyright (c) 2000-2015 the FFmpeg
developers
built with gcc 4.7 (SUSE Linux)
configuration: --enable-gpl
libavutil 54. 25.100 / 54. 25.100
libavcodec 56. 41.100 / 56. 41.100
libavformat 56. 33.101 / 56. 33.101
libavdevice 56. 4.100 / 56. 4.100
libavfilter 5. 16.101 / 5. 16.101
libswscale 3. 1.101 / 3. 1.101
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 3.100 / 53. 3.100
Input #0, avi, from 'untitled.split.2_new.split.1.cut.avi':
Metadata:
encoder : Lavf56.19.100
Duration: 00:00:00.08, start: 0.000000, bitrate: 112165 kb/s
Stream #0:0: Video: ffv1 (FFV1 / 0x31564646), yuv420p10le, 1280x720,
SAR 1:1 DAR 16:9, 23.98 fps, 23.98 tbr, 23.98 tbn, 23.98 tbc
[New Thread 0x7ffff14f0700 (LWP 14803)]
[New Thread 0x7ffff0cef700 (LWP 14804)]
[New Thread 0x7ffff04ee700 (LWP 14805)]
[New Thread 0x7fffefced700 (LWP 14806)]
[New Thread 0x7fffef4ec700 (LWP 14807)]
[New Thread 0x7fffeeceb700 (LWP 14808)]
[New Thread 0x7fffee4ea700 (LWP 14809)]
[New Thread 0x7fffedce9700 (LWP 14810)]
[New Thread 0x7fffed4e8700 (LWP 14811)]
[New Thread 0x7fffecce7700 (LWP 14812)]
[New Thread 0x7fffec4e6700 (LWP 14813)]
[New Thread 0x7fffebce5700 (LWP 14814)]
[New Thread 0x7fffeb4e4700 (LWP 14815)]
[New Thread 0x7fffeace3700 (LWP 14816)]
[New Thread 0x7fffea4e2700 (LWP 14817)]
[New Thread 0x7fffe9ce1700 (LWP 14818)]
[New Thread 0x7fffe94e0700 (LWP 14819)]
[New Thread 0x7fffe8cdf700 (LWP 14820)]
Output #0, null, to 'pipe:':
Metadata:
encoder : Lavf56.33.101
Stream #0:0: Video: rawvideo (Y3[11][10] / 0xA0B3359), yuv420p10le,
1280x720 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 23.98 fps, 23.98 tbn, 23.98
tbc
Metadata:
encoder : Lavc56.41.100 rawvideo
Stream mapping:
Stream #0:0 -> #0:0 (ffv1 (native) -> rawvideo (native))
Press [q] to stop, [?] for help
[null @ 0x1b9f120] Encoder did not produce proper pts, making some up.
Program received signal SIGSEGV, Segmentation fault.
ff_hqdn3d_row_10_x86.loop2 () at libavfilter/x86/vf_hqdn3d.asm:105
105 HQDN3D_ROW 10
(gdb) bt
#0 ff_hqdn3d_row_10_x86.loop2 () at libavfilter/x86/vf_hqdn3d.asm:105
#1 0x000000000247b740 in ?? ()
#2 0x00000000004da245 in denoise_spatial (temporal=0x1b994a0,
spatial=0x1b95460,
depth=10, dstride=2560, sstride=2560, h=720, w=1280,
frame_ant=0xfffffffd,
line_ant=0x1ba10c0, dst=<optimized out>, src=<optimized out>,
s=0x1ba0cc0)
at libavfilter/vf_hqdn3d.c:107
#3 denoise_depth (depth=10, temporal=<optimized out>, spatial=0x1b93460,
dstride=2560,
sstride=2560, h=<optimized out>, w=<optimized out>,
frame_ant_ptr=0x1ba0cf0,
line_ant=0x1ba10c0, dst=<optimized out>, src=<optimized out>,
s=0x1ba0cc0)
at libavfilter/vf_hqdn3d.c:147
#4 filter_frame (inlink=inlink at entry=0x1b9cde0, in=0x25dc060)
at libavfilter/vf_hqdn3d.c:312
#5 0x00000000004a1ece in ff_filter_frame_framed
(link=link at entry=0x1b9cde0,
frame=0x271f81e, frame at entry=0x25dc060) at libavfilter/avfilter.c:1091
#6 0x00000000004a2f59 in ff_filter_frame (link=link at entry=0x1b9cde0,
frame=0x25dc060)
at libavfilter/avfilter.c:1172
#7 0x00000000004a72f2 in request_frame (link=0x1b9cde0) at
libavfilter/buffersrc.c:500
#8 0x00000000004a758b in av_buffersrc_add_frame_internal
(ctx=ctx at entry=0x1b92740,
frame=frame at entry=0x1ff6840, flags=flags at entry=4) at
libavfilter/buffersrc.c:181
#9 0x00000000004a791d in av_buffersrc_add_frame_flags (ctx=0x1b92740,
frame=frame at entry=0x1ff6840, flags=flags at entry=4) at
libavfilter/buffersrc.c:106
#10 0x0000000000489832 in decode_video (ist=ist at entry=0x1b9e780,
pkt=pkt at entry=0x7fffffffd670,
got_output=got_output at entry=0x7fffffffd66c)
at ffmpeg.c:2098
#11 0x000000000048b56a in process_input_packet (ist=0x1b9e780, pkt=0x0) at
ffmpeg.c:2232
#12 0x000000000048efbd in process_input (file_index=0) at ffmpeg.c:3567
#13 transcode_step () at ffmpeg.c:3847
#14 transcode () at ffmpeg.c:3900
#15 0x00000000004726fb in main (argc=<optimized out>, argv=0x7fffffffdce8)
at ffmpeg.c:4082
(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x548652 to 0x548692:
0x0000000000548652 <ff_hqdn3d_row_10_x86.loop2+20>: mov
%bx,(%rdx,%r8,2)
0x0000000000548657 <ff_hqdn3d_row_10_x86.loop2+25>: sub %r11,%r10
0x000000000054865a <ff_hqdn3d_row_10_x86.loop2+28>: sar $0x4,%r10
0x000000000054865e <ff_hqdn3d_row_10_x86.loop2+32>: movswl
(%r9,%r10,2),%r10d
0x0000000000548663 <ff_hqdn3d_row_10_x86.loop2+37>: add %r11d,%r10d
0x0000000000548666 <ff_hqdn3d_row_10_x86.loop2+40>: movzwl
(%rcx,%r8,2),%r11d
0x000000000054866b <ff_hqdn3d_row_10_x86.loop2+45>: sub %rbx,%r11
0x000000000054866e <ff_hqdn3d_row_10_x86.loop2+48>: sar $0x4,%r11
=> 0x0000000000548672 <ff_hqdn3d_row_10_x86.loop2+52>: movswl
(%rax,%r11,2),%r11d
0x0000000000548677 <ff_hqdn3d_row_10_x86.loop2+57>: add %ebx,%r11d
0x000000000054867a <ff_hqdn3d_row_10_x86.loop2+60>: mov
%r11w,(%rcx,%r8,2)
0x000000000054867f <ff_hqdn3d_row_10_x86.loop2+65>: shr $0x6,%r11d
0x0000000000548683 <ff_hqdn3d_row_10_x86.loop2+69>: mov
%r11w,(%rsi,%r8,2)
0x0000000000548688 <ff_hqdn3d_row_10_x86.loop2+74>: inc %r8
0x000000000054868b <ff_hqdn3d_row_10_x86.loop2+77>: jl 0x548630
<ff_hqdn3d_row_10_x86.loop>
0x000000000054868d <ff_hqdn3d_row_10_x86.loop2+79>: je 0x54863e
<ff_hqdn3d_row_10_x86.loop2>
0x000000000054868f <ff_hqdn3d_row_10_x86.loop2+81>: pop %rbx
0x0000000000548690 <ff_hqdn3d_row_10_x86.loop2+82>: retq
0x0000000000548691 <ff_hqdn3d_row_10_x86.loop2+83>: data32 data32
data32 data32 data32 nopw %cs:0x0(%rax,%rax,1)
End of assembler dump.
(gdb) info register
rax 0x1b994a0 28939424
rbx 0xfffffffd 4294967293
rcx 0x247c13e 38256958
rdx 0x1ba1abe 28973758
rsi 0x271f81e 41023518
rdi 0x7fffdc143c1e 140736885701662
rbp 0x4ff 0x4ff
rsp 0x7fffffffd240 0x7fffffffd240
r8 0xffffffffffffff61 -159
r9 0x1b95460 28922976
r10 0x6a 106
r11 0xfffffffff0000002 -268435454
r12 0x203 515
r13 0x1ba10c0 28971200
r14 0x1b95460 28922976
r15 0x1b994a0 28939424
rip 0x548672 0x548672 <ff_hqdn3d_row_10_x86.loop2+52>
eflags 0x10282 [ SF IF RF ]
cs 0x33 51
ss 0x2b 43
ds 0x0 0
es 0x0 0
fs 0x0 0
gs 0x0 0
}}}
Not reproducible without yasm / nasm.
--
Ticket URL: <https://trac.ffmpeg.org/ticket/4301#comment:3>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker
More information about the FFmpeg-trac
mailing list