[FFmpeg-devel] [PATCH 5/9] x86: simple_idct10_template: fix overflow in pass

[Michael Niedermayer]

On Mon, Oct 12, 2015 at 07:37:46PM +0200, Christophe Gisquet wrote:
> When the input of a pass has 15 or 16 bits of precision (in particular
> the column pass), the addition of a bias to W4 may lead to overflows
> in the input to pmaddwd.
> 
> This requires postponing the adding of the bias to after the first
> butterfly. To do so, the fact that m15, unused although zeroed, is
> exploited. In case the pass is safe, an address can be directly used,
> and the number of xmm regs can be decreased. Otherwise, the 32bits bias
> is loaded into it.
> ---
>  libavcodec/x86/proresdsp.asm              |  8 ++++----
>  libavcodec/x86/simple_idct10_template.asm | 13 ++++++++++++-
>  2 files changed, 16 insertions(+), 5 deletions(-)

how can i reproduce these overflows ?

[...]
-- 
Michael     GnuPG fingerprint: 9FF2128B147EF6730BADF133611EC787040B0FAB

Opposition brings concord. Out of discord comes the fairest harmony.
-- Heraclitus
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 181 bytes
Desc: Digital signature
URL: <http://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20151013/d19efec2/attachment.sig>