[FFmpeg-trac] #8505(avcodec:open): fast aac encoder produces invalid output for x86 32-bit builds

[FFmpeg]

#8505: fast aac encoder produces invalid output for x86 32-bit builds
-------------------------------------+-------------------------------------
             Reporter:               |                    Owner:
  LeadAssimilator                    |
                 Type:  defect       |                   Status:  open
             Priority:  important    |                Component:  avcodec
              Version:  git-master   |               Resolution:
             Keywords:  regression   |               Blocked By:
  aac                                |
             Blocking:               |  Reproduced by developer:  1
Analyzed by developer:  0            |
-------------------------------------+-------------------------------------

Comment (by LeadAssimilator):

 gcc only uses sse/sse2 if it is explicitly told to via -msse or -msse2 and
 -mfpmath=sse.  By default, ffmpeg will not compile with these flags unless
 they are set via --extra-cflags and -extra-cxxflags.

 Introducing an sse2 optimized version of quant would help in the common
 case, but I also spotted that compiling with -O2 causes the C version of
 quantize_bands to output 16 for the input in question, but quant still
 outputs 15.  So that means compiling with -O2 and running on a system
 without sse2 will also have this same problem.  While rare these days, it
 is still technically wrong and a new sse2 optimized version of quant
 wouldn't help there.

 While clamping is sort of a hack, it might be the best option to prevent
 corrupted output in all cases, even for other cpu architectures where
 similar bugs may exist.  It could still be combined with an sse2 optimized
 version of quant though to improve correctness in the common case.

--
Ticket URL: <https://trac.ffmpeg.org/ticket/8505#comment:21>
FFmpeg <https://ffmpeg.org>
FFmpeg issue tracker