[FFmpeg-devel] [PATCH] x86inc: support stack mem allocation and re-alignment in PROLOGUE.

[Ronald S. Bultje]

Hi,

On Sun, Dec 16, 2012 at 10:16 AM, Alexander Strasser <eclipse7 at gmx.net> wrote:
> Hi Ronald,
>
> Alexander Strasser wrote:
>> Ronald S. Bultje wrote:
> [...]
>> >
>> > http://lists.libav.org/pipermail/libav-devel/2012-December/040350.html
>> >
>> > Fixes it.
>>
>>   Thanks. I will probably not be able to test on cygwin for the next few
>> days, but I will test with the fix as soon as I can.
>
>   Now I get a different error:
>
>   libavcodec/x86/h264_deblock.asm:1025: error: (ASSERT:2) assert failed
>
>   I think you hit that error before when working on ASM code. But might
> have been triggered by something completely different.
>
>   I for my part do not know how to proceed in figuring out why it fails
> this time.

I've fixed that in x264, see bottom part of this patch:
http://privatepaste.com/81432e7169

Basically:

@@ -541,12 +547,15 @@ DECLARE_ARG 7, 8, 9, 10, 11, 12, 13, 14
 %macro PROLOGUE 2-5+ ; #args, #regs, #xmm_regs, [stack_size,] arg_names...
     %assign num_args %1
     %assign regs_used %2
+    ASSERT regs_used >= num_args
+    %if num_args > 7
+        %assign num_args 7
+    %endif
     %if regs_used > 7
         %assign regs_used 7
     %endif
     SETUP_STACK_POINTER %4
     ASSERT regs_used <= 7
-    ASSERT regs_used >= num_args
     PUSH_IF_USED 3, 4, 5, 6
     ALLOC_STACK %4
     LOAD_IF_USED 0, 1, 2, 3, 4, 5, 6

Ronald