C Calling Conventions

In order to make any sense of C functions at the assembly level, you need to know the calling convention used by the function.  A calling convention specifies how arguments are passed to the function, via machine-level registers or via the stack.  Because a calling convention deals with operations at the machine-level, it is necessarily architecture specific and closely related to the target architecture’s Application Binary Interface (ABI).  Calling conventions can also be specific to compilers, but most compilers support at least a few of the standard calling conventions.  Let’s focus on the x86 architecture and the two most popular calling conventions: cdecl and stdcall.

Similarities

There are many similarities between cdecl and stdcall.  Both conventions pass arguments on the stack and push the arguments onto the stack in order from right to left.  For example, for a function with the following signature

int foo(int a, int b, int c)

the argument c is pushed first (appearing at a higher address for a stack that grows downward), followed by argument b, followed by argument a (at the lowest address for a stack that grows downward).

Another similarity is that, for both calling conventions, the registers EAX, ECX, and EDX are preserved by the caller and available for immediate use by the called function.

Finally, in both conventions, the return value is passed back in EAX.

Differences
So what’s the difference?  The main difference between these two calling conventions is who cleans up the stack, the caller or the callee.  In the case of cdecl, the caller cleans up the stack.  In the case of stdcall, the callee cleans up the stack.
Examples
Let’s look at a few examples using a simple C function with the following function prototype:
int add(int a, int b, int c);
The assembly listing below shows the above function using the cdecl calling convention.  Notice that the callee does nothing to clean up the stack and leaves that to the caller.
cdecl example

cdecl example

 

In contrast, the assembly listing below shows the same function using the stdcall calling convention.  In this case, the callee cleans up the stack by popping 12 bytes off the stack as part of the return instruction.

stdcall example

stdcall example

 

Posted in Disassembly Tidbits, Uncategorized

Leave a Reply

Your email address will not be published. Required fields are marked *

*


nine − = 8

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>