[FFmpeg-devel] [PATCH] libswscale/x86/yuv2rgb: Add missing EMMS

[Mario Hros]

Ping?

Not including EMMS causes the FPU to become unusable (as the IE and SF bits are set in the status word). This breaks code which, for instance, calls fmod() and is compiled by GCC with the -O3 flag enabled. In that scenario, GCC implements fmod using the FPREM FPU instruction, but because FPU is in the invalid state at that point, the result is NaN.

This must to breaking other code using FPU also. The original code with inline assembly used EMMS at the end.
________________________________
From: Mario Hros <k3x-devel at outlook.com>
Sent: Wednesday, June 5, 2024 7:35 PM
To: ffmpeg-devel at ffmpeg.org <ffmpeg-devel at ffmpeg.org>
Cc: Mario Hros <k3x-devel at outlook.com>
Subject: [PATCH] libswscale/x86/yuv2rgb: Add missing EMMS

Previous rewrite from inline assembly into nasm (commit e934194) missed the required EMMS instruction to bring the x87 FPU back into usable state.

Signed-off-by: Mario Hros <k3x-devel at outlook.com>
---
 libswscale/x86/yuv_2_rgb.asm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/libswscale/x86/yuv_2_rgb.asm b/libswscale/x86/yuv_2_rgb.asm
index e3470fd9ad..7a247797e4 100644
--- a/libswscale/x86/yuv_2_rgb.asm
+++ b/libswscale/x86/yuv_2_rgb.asm
@@ -353,6 +353,7 @@ cglobal %1_420_%2%3, GPR_num, GPR_num, reg_num, parameters
 add imageq, 8 * depth * time_num
 add indexq, 4 * time_num
 js .loop0
+emms

 RET

--
2.39.3 (Apple Git-146)