[FFmpeg-devel] [PATCH] x86inc: support stack mem allocation and re-alignment in PROLOGUE.

[Alexander Strasser]

Ronald S. Bultje wrote:
> On Sun, Dec 16, 2012 at 10:16 AM, Alexander Strasser <eclipse7 at gmx.net> wrote:
> > Alexander Strasser wrote:
> >>
> >>   Thanks. I will probably not be able to test on cygwin for the next few
> >> days, but I will test with the fix as soon as I can.
> >
> >   Now I get a different error:
> >
> >   libavcodec/x86/h264_deblock.asm:1025: error: (ASSERT:2) assert failed
> >
> >   I think you hit that error before when working on ASM code. But might
> > have been triggered by something completely different.
> >
> >   I for my part do not know how to proceed in figuring out why it fails
> > this time.
> 
> I've fixed that in x264, see bottom part of this patch:
> http://privatepaste.com/81432e7169
> 
> Basically:
> 
> @@ -541,12 +547,15 @@ DECLARE_ARG 7, 8, 9, 10, 11, 12, 13, 14
>  %macro PROLOGUE 2-5+ ; #args, #regs, #xmm_regs, [stack_size,] arg_names...
>      %assign num_args %1
>      %assign regs_used %2
> +    ASSERT regs_used >= num_args
> +    %if num_args > 7
> +        %assign num_args 7
> +    %endif
>      %if regs_used > 7
>          %assign regs_used 7
>      %endif
>      SETUP_STACK_POINTER %4
>      ASSERT regs_used <= 7
> -    ASSERT regs_used >= num_args
>      PUSH_IF_USED 3, 4, 5, 6
>      ALLOC_STACK %4
>      LOAD_IF_USED 0, 1, 2, 3, 4, 5, 6

  This helps to some extent but I get an error with cpuid.asm now:

  libavutil/x86/cpuid.asm:70: fatal: (cglobal:2) No rvalue found on pasting

  Maybe I did something wrong, didn't have time to investigate further.


  Alexander

P.S.
I did this modification based on your explanation:
diff --git a/libavutil/x86/x86inc.asm b/libavutil/x86/x86inc.asm
index 59ddc9d..1505756 100644
--- a/libavutil/x86/x86inc.asm
+++ b/libavutil/x86/x86inc.asm
@@ -568,12 +568,15 @@ DECLARE_ARG 7, 8, 9, 10, 11, 12, 13, 14
 %macro PROLOGUE 2-5+ ; #args, #regs, #xmm_regs, [stack_size,] arg_names...
     %assign num_args %1
     %assign regs_used %2
+    ASSERT regs_used >= num_args
+    %if num_args > 7
+        %assign num_args 7
+    %endif
     %if regs_used > 7
         %assign regs_used 7
     %endif
     SETUP_STACK_POINTER %4
     ASSERT regs_used <= 7
-    ASSERT regs_used >= num_args
     PUSH_IF_USED 3, 4, 5, 6
     ALLOC_STACK %4
     LOAD_IF_USED 0, 1, 2, 3, 4, 5, 6