[FFmpeg-trac] #5231(avcodec:closed): Crashes in ff_deblock_v_luma_8_sse2

FFmpeg trac at avcodec.org
Fri Oct 7 23:17:22 EEST 2016

#5231: Crashes in ff_deblock_v_luma_8_sse2
             Reporter:  mi           |                    Owner:
                 Type:  defect       |                   Status:  closed
             Priority:  important    |                Component:  avcodec
              Version:  git-master   |               Resolution:  fixed
             Keywords:  h264 crash   |               Blocked By:
  SIGSEGV                            |  Reproduced by developer:  1
             Blocking:               |
Analyzed by developer:  0            |

Comment (by cehoyos):

 Replying to [comment:25 mi]:
 > Replying to [comment:24 cehoyos]:
 > {{{
 > --- a/configure
 > +++ b/configure
 > @@ -6202,6 +6202,7 @@ elif enabled llvm_gcc; then
 >      check_cflags -mllvm -stack-alignment=16
 >  elif enabled clang; then
 >      check_cflags -mllvm -stack-alignment=16
 > +    check_cflags -mstack-alignment=16
 >      check_cflags -Qunused-arguments
 >      check_cflags -Werror=implicit-function-declaration
 >      check_cflags -Werror=missing-prototypes
 > }}}
 > Does this not cause a pessimization in other parts of the code, though?

 I don't think so, as explained by the BSD developers, this is the default
 on (all) other systems, it was also always meant to be used (see the line
 {{{-mllvm -stack-alignment=16}}} above).

 > That was my worry -- that by setting the stack-alignment to a non-
 default value to solve a problem in one function slows things down (or
 increases memory use) everywhere else...

 That would be true for {{{-mstackrealign}}} which I fear may be necessary
 to fix the original issue with firefox;-(
 As said, I did not test with firefox.

 > > Thanks to Mikhail
 > You are welcome. I'll keep your account on the box active. In addition
 to clang-3.4.1 (a.k.a. `cc`) and gcc-4.2.1 (a.k.a. `gcc`), it has the
 latest versions of clang-3.6 (`clang36`), 3.7 (`clang37`), 3.8 (`clang38`)

 I only tested clang 3.8 (and gcc 4.2, see below).

 > and gcc-6 (`gcc6`).

 I had not found this one;-(
 It is the only compiler that really works for FFmpeg on FreeBSD (gcc 4.2
 is broken, configure warns about it)

 Ideally, you would first test with clang and {{{--extra-cflags=-mstack-
 alignment=16}}}, if this does not help, you can try {{{--cc=gcc6}}}. If
 both still crash with firefox, one alternative is to also compile firefox
 with increased stack alignment.

Ticket URL: <https://trac.ffmpeg.org/ticket/5231#comment:26>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker

More information about the FFmpeg-trac mailing list