[Libav-user] avcodec_decode_video2 crash with HEVC and get_buffer2

Никита Скиба zaulan at gmail.com
Fri Mar 11 14:11:21 CET 2016


Hello again!

I have updated libs to
ffmpeg version N-78992-gac19746 Copyright (c) 2000-2016 the FFmpeg
developers
built with gcc 4.9.2 (GCC)
configuration: --cross-prefix=x86_64-w64-mingw32- --arch=x86
--target-os=mingw32
 --enable-debug=3 --disable-optimizations --disable-stripping
--enable-memalign-
hack --enable-shared --disable-static --disable-dxva2
--extra-cflags=-U__STRICT_
ANSI__ --extra-ldflags=-static-libgcc
--prefix=/home/nikita.skiba/build-3.0.1 --
shlibdir=/home/nikita.skiba/build-3.0.1/bin/x86_64
--build-suffix=-itv-3.0.1 --b
indir=/home/nikita.skiba/build-3.0.1/bin/x86_64
--libdir=/home/nikita.skiba/buil
d-3.0.1/lib/x86_64 --enable-runtime-cpudetect --enable-w32threads
libavutil      55. 19.100 / 55. 19.100
libavcodec     57. 28.100 / 57. 28.100
libavformat    57. 28.100 / 57. 28.100
libavdevice    57.  0.101 / 57.  0.101
libavfilter     6. 39.102 /  6. 39.102
libswscale      4.  0.100 /  4.  0.100
libswresample   2.  0.101 /  2.  0.101

Also, I rebuild lib with debug information support and make possible to get
backtrace of segfault via gdb.
Backtrace
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 4764.0x268c]
0x000007fecdd455bf in ff_hevc_transform_add32_8_avx2 ()
   from C:\Program Files\AxxonSoft\AxxonSmart\bin\avcodec-itv-3.0.1-57.dll
(gdb) bt
#0  0x000007fecdd455bf in ff_hevc_transform_add32_8_avx2 ()
   from C:\Program Files\AxxonSoft\AxxonSmart\bin\avcodec-itv-3.0.1-57.dll
#1  0x000007fecd6e9056 in ff_hevc_hls_residual_coding (s=0x26dfdec0, x0=0,
    y0=0, log2_trafo_size=5, scan_idx=SCAN_DIAG, c_idx=0)
    at libavcodec/hevc_cabac.c:1513
#2  0x000007fecd6d9f74 in hls_transform_unit (s=0x26dfdec0, x0=0, y0=0,
    xBase=0, yBase=0, cb_xBase=0, cb_yBase=0, log2_cb_size=6,
    log2_trafo_size=5, blk_idx=0, cbf_luma=1, cbf_cb=0x2fbbcb10,
    cbf_cr=0x2fbbcb00) at libavcodec/hevc.c:1019
#3  0x000007fecd6db263 in hls_transform_tree (s=0x26dfdec0, x0=0, y0=0,
    xBase=0, yBase=0, cb_xBase=0, cb_yBase=0, log2_cb_size=6,
    log2_trafo_size=5, trafo_depth=1, blk_idx=0, base_cbf_cb=0x2fbbcbf0,
    base_cbf_cr=0x2fbbcbe0) at libavcodec/hevc.c:1253
#4  0x000007fecd6dafde in hls_transform_tree (s=0x26dfdec0, x0=0, y0=0,
    xBase=0, yBase=0, cb_xBase=0, cb_yBase=0, log2_cb_size=6,
    log2_trafo_size=6, trafo_depth=0, blk_idx=0,
    base_cbf_cb=0x7fecdf5f500 <cbf>, base_cbf_cr=0x7fecdf5f500 <cbf>)
    at libavcodec/hevc.c:1235
#5  0x000007fecd6df86c in hls_coding_unit (s=0x26dfdec0, x0=0, y0=0,
    log2_cb_size=6) at libavcodec/hevc.c:2142
#6  0x000007fecd6dfe5f in hls_coding_quadtree (s=0x26dfdec0, x0=0, y0=0,
    log2_cb_size=6, cb_depth=0) at libavcodec/hevc.c:2239
#7  0x000007fecd6e08e5 in hls_decode_entry (avctxt=0x878260,
    isFilterThread=0x2fbbcea0) at libavcodec/hevc.c:2344
#8  0x000007fecda20266 in avcodec_default_execute (c=0x878260,
    func=0x7fecd6e05d1 <hls_decode_entry>, arg=0x2fbbcea0, ret=0x2fbbce90,
    count=1, size=4) at libavcodec/utils.c:990
#9  0x000007fecd6e0a1a in hls_slice_data (s=0x26dfdec0)
    at libavcodec/hevc.c:2371
#10 0x000007fecd6e2558 in decode_nal_unit (s=0x26dfdec0, nal=0x26ce3c50)
    at libavcodec/hevc.c:2828
#11 0x000007fecd6e279e in decode_nal_units (s=0x26dfdec0, buf=0x45ee0410 "",
    length=7130) at libavcodec/hevc.c:2885
#12 0x000007fecd6e2c19 in hevc_decode_frame (avctx=0x878260,
data=0x26d7f020,
    got_output=0x2fbbd0d8, avpkt=0x2fbbd010) at libavcodec/hevc.c:2989
#13 0x000007fecda23d20 in avcodec_decode_video2 (avctx=0x878260,
    picture=0x26d7f020, got_picture_ptr=0x2fbbd0d8, avpkt=0x2fbbd310)
    at libavcodec/utils.c:2172


(gdb) disass $pc-32,$pc+32
Dump of assembler code from 0x7fecdd4559f to 0x7fecdd455df:
   0x000007fecdd4559f <ff_hevc_transform_add32_8_avx2+79>:      stc
   0x000007fecdd455a0 <ff_hevc_transform_add32_8_avx2+80>:      fadd
%st,%st(5)
   0x000007fecdd455a2 <ff_hevc_transform_add32_8_avx2+82>:      std
   0x000007fecdd455a3 <ff_hevc_transform_add32_8_avx2+83>:      stc
   0x000007fecdd455a4 <ff_hevc_transform_add32_8_avx2+84>:      out
 %al,(%dx)
   0x000007fecdd455a5 <ff_hevc_transform_add32_8_avx2+85>:      vpackuswb
%ymm6,%ymm4,%ymm4
   0x000007fecdd455a9 <ff_hevc_transform_add32_8_avx2+89>:      vpackuswb
%ymm5,%ymm3,%ymm3
   0x000007fecdd455ad <ff_hevc_transform_add32_8_avx2+93>:      vpaddusb
(%rcx),%ymm2,%ymm2
   0x000007fecdd455b1 <ff_hevc_transform_add32_8_avx2+97>:      vpaddusb
(%rcx,%r8,1),%ymm4,%ymm4
   0x000007fecdd455b7 <ff_hevc_transform_add32_8_avx2+103>:     vpsubusb
%ymm1,%ymm2,%ymm2
   0x000007fecdd455bb <ff_hevc_transform_add32_8_avx2+107>:     vpsubusb
%ymm3,%ymm4,%ymm4
=> 0x000007fecdd455bf <ff_hevc_transform_add32_8_avx2+111>:     vmovdqa
%ymm2,(%rcx)
   0x000007fecdd455c3 <ff_hevc_transform_add32_8_avx2+115>:     vmovdqa
%ymm4,(%rcx,%r8,1)
   0x000007fecdd455c9 <ff_hevc_transform_add32_8_avx2+121>:     vmovdqa
0x80(%rdx),%xmm2
   0x000007fecdd455d1 <ff_hevc_transform_add32_8_avx2+129>:     vmovdqa
0x90(%rdx),%xmm6
   0x000007fecdd455d9 <ff_hevc_transform_add32_8_avx2+137>:     vinserti128
$0x1,0xa0(%rdx),%ymm2,%ymm2
End of assembler dump.

(gdb) info all-registers
rax            0x7fecdd45550    8790956332368
rbx            0x23fe5ad0       603871952
rcx            0x47ee0410       1206780944
rdx            0x26e015a0       652219808
rsi            0x3e96f280       1050079872
rdi            0x2fbbc980       800835968
rbp            0x2fbbc570       0x2fbbc570
rsp            0x2fbbc4e8       0x2fbbc4e8
r8             0x780    1920
r9             0x1680   5760
r10            0x76     118
r11            0xa      10
r12            0x2fbbf9e0       800848352
r13            0x1      1
r14            0x1      1
r15            0x1      1
rip            0x7fecdd455bf    0x7fecdd455bf
<ff_hevc_transform_add32_8_avx2+11
1>
eflags         0x10202  [ IF RF ]
cs             0x33     51
ss             0x2b     43
ds             0x0      0
es             0x0      0
fs             0x0      0
gs             0x0      0
st0            0        (raw 0x00000000000000000000)
st1            0        (raw 0x00000000000000000000)
st2            0        (raw 0x00000000000000000000)
st3            0        (raw 0x00000000000000000000)
st4            0        (raw 0x00000000000000000000)
st5            0        (raw 0x00000000000000000000)
st6            0        (raw 0x00000000000000000000)
st7            1        (raw 0x3fff8000000000000000)
fctrl          0x27f    639
fstat          0x0      0
ftag           0x0      0
fiseg          0x0      0
fioff          0xcc3e1dae       -868344402
foseg          0x0      0
fooff          0x2fbbca18       800836120
fop            0x0      0
xmm0           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm1           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {
    0x8000000000000000, 0x8000000000000000}, v16_int8 = {
    0x6f <repeats 16 times>}, v8_int16 = {0x6f6f, 0x6f6f, 0x6f6f, 0x6f6f,
    0x6f6f, 0x6f6f, 0x6f6f, 0x6f6f}, v4_int32 = {0x6f6f6f6f, 0x6f6f6f6f,
    0x6f6f6f6f, 0x6f6f6f6f}, v2_int64 = {0x6f6f6f6f6f6f6f6f,
    0x6f6f6f6f6f6f6f6f}, uint128 = 0x6f6f6f6f6f6f6f6f6f6f6f6f6f6f6f6f}
xmm2           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x11 <repeats 16 times>}, v8_int16 = {0x1111, 0x1111, 0x1111,
    0x1111, 0x1111, 0x1111, 0x1111, 0x1111}, v4_int32 = {0x11111111,
    0x11111111, 0x11111111, 0x11111111}, v2_int64 = {0x1111111111111111,
    0x1111111111111111}, uint128 = 0x11111111111111111111111111111111}
xmm3           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {
    0x8000000000000000, 0x8000000000000000}, v16_int8 = {
    0x6f <repeats 16 times>}, v8_int16 = {0x6f6f, 0x6f6f, 0x6f6f, 0x6f6f,
    0x6f6f, 0x6f6f, 0x6f6f, 0x6f6f}, v4_int32 = {0x6f6f6f6f, 0x6f6f6f6f,
    0x6f6f6f6f, 0x6f6f6f6f}, v2_int64 = {0x6f6f6f6f6f6f6f6f,
    0x6f6f6f6f6f6f6f6f}, uint128 = 0x6f6f6f6f6f6f6f6f6f6f6f6f6f6f6f6f}
xmm4           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x11 <repeats 16 times>}, v8_int16 = {0x1111, 0x1111, 0x1111,
    0x1111, 0x1111, 0x1111, 0x1111, 0x1111}, v4_int32 = {0x11111111,
    0x11111111, 0x11111111, 0x11111111}, v2_int64 = {0x1111111111111111,
    0x1111111111111111}, uint128 = 0x11111111111111111111111111111111}
xmm5           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x6f, 0x0, 0x6f, 0x0, 0x6f, 0x0, 0x6f, 0x0, 0x6f, 0x0, 0x6f,
    0x0, 0x6f, 0x0, 0x6f, 0x0}, v8_int16 = {0x6f, 0x6f, 0x6f, 0x6f, 0x6f,
    0x6f, 0x6f, 0x6f}, v4_int32 = {0x6f006f, 0x6f006f, 0x6f006f, 0x6f006f},
  v2_int64 = {0x6f006f006f006f, 0x6f006f006f006f},
  uint128 = 0x006f006f006f006f006f006f006f006f}
xmm6           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {
    0x8000000000000000, 0x8000000000000000}, v16_int8 = {0x91, 0xff, 0x91,
    0xff, 0x91, 0xff, 0x91, 0xff, 0x91, 0xff, 0x91, 0xff, 0x91, 0xff, 0x91,
    0xff}, v8_int16 = {0xff91, 0xff91, 0xff91, 0xff91, 0xff91, 0xff91,
    0xff91, 0xff91}, v4_int32 = {0xff91ff91, 0xff91ff91, 0xff91ff91,
    0xff91ff91}, v2_int64 = {0xff91ff91ff91ff91, 0xff91ff91ff91ff91},
  uint128 = 0xff91ff91ff91ff91ff91ff91ff91ff91}
xmm7           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm8           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm9           {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm10          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm11          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm12          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm13          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm14          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
xmm15          {v4_float = {0x0, 0x0, 0x0, 0x0}, v2_double = {0x0, 0x0},
  v16_int8 = {0x0 <repeats 16 times>}, v8_int16 = {0x0, 0x0, 0x0, 0x0, 0x0,
    0x0, 0x0, 0x0}, v4_int32 = {0x0, 0x0, 0x0, 0x0}, v2_int64 = {0x0, 0x0},
  uint128 = 0x00000000000000000000000000000000}
mxcsr          0x1fa0   [ PE IM DM ZM OM UM PM ]

Could you please help? Or at least point me to the possible cause of the
segfault? I could also provide HEVC sample if you need it.

Thank in advance,
Nikita

2016-03-09 17:49 GMT+03:00 Carl Eugen Hoyos <cehoyos at ag.or.at>:

> Никита Скиба <zaulan at ...> writes:
>
> > I have updated libav dlls to
> >
> > ffmpeg version N-76467-ga8b254e Copyright (c) 2000-2015 the FFmpeg
>
> This is 2500 changes old and not supported.
>
> You may want to provide a backtrace and please make
> sure that your input buffers are padded as required
> by libavcodec.
>
> Sorry, Carl Eugen
> _______________________________________________
> Libav-user mailing list
> Libav-user at ffmpeg.org
> http://ffmpeg.org/mailman/listinfo/libav-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://ffmpeg.org/pipermail/libav-user/attachments/20160311/26e08913/attachment.html>


More information about the Libav-user mailing list