[FFmpeg-devel] [WIP][PATCH]v3 Opus Pyramid Vector Quantization Search in x86 SIMD asm

[Ivan Kalvachev]

First, I've removed the hacked formula.

While it seems to improve precision in the synthetic test,
it is not enough to avoid assigning pulses in padded area.
Worse, it also interferes with the special case with
subtraction of pulses from y[i]==0 output. Handling these cases
needs combining two masks and using "blendvps" instead the faster "maxps".
Most of these problems are related to the fact that
the hacked formula result is in the range -1.0 to 0.0
where 0.0 is the perfect match.

Second, fixed the all_float rounding mode,
I got it swapped in v2.

Third, reordered some code. Moved the branch
for exact match of the pre-search, to be first taken.
Load the constants needed in the pulse search
only when they will be used.

Use a macro for conditional loading/relabeling of the constants.
Use similar idea for loading and using PIC register.

Use "smartalign" on NASM, to avoid 15 consequitev 1 byte NOPs.
YASM is smart by default.

I attach a second patch, that is
slightly modified version of atomnuker's code.
It sums the distortions of C and SIMD implementations.

In my test approx#2 seems to be pretty close to the C version,
sometimes better, sometimes worse.
If you find a file with dramatic difference, I'm interested. :D

If there are no more issues, v4 would be cleaned up and ready for review.

Best Regards.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-SIMD-opus-pvq_search-implementation-v3.patch
Type: text/x-patch
Size: 27958 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170701/3d33a79e/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Code-to-measure-the-total-distortion-sum.patch
Type: text/x-patch
Size: 2785 bytes
Desc: not available
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20170701/3d33a79e/attachment-0001.bin>