first glance of alloca in ctf

0x1 background

when I was doing zeropts CTF 2022, I found a pwn question called accountant.

There is a line of code that use alloca to allocate memory. However, at that time, I didn't know that alloca allocate the memory on the stack. so I simply thought there is no bug... :(

later, Super Guesser publish their writeup on accountant that point out that alloca allocate space on the stack.

So I could have a chance of looking at alloca.

0x2 basic idea

The alloca() function allocates size bytes of space in the stack frame of the caller. This temporary space is automatically freed when the function that called alloca() returns to its caller.
https://www.mkssoftware.com/docs/man3/alloca.3.asp

Basically, alloca allocate a space on the stack instead of heap.

Lets do some code to find out what alloca do. above is a example program.

#include <stdio.h>
#include <stdlib.h>
void test() {
    long * ptr1 = (long *)0x1000;
    long * ptr2 = (long *)0x1001;
    // just some offset to make it clear
    long x = 0x1002;
    // lets alloca some space!
    // normal alloca
    ptr1 = (long *) alloca(sizeof(long));
    *ptr1 = 0x2001;
    ptr1 = (long *) alloca(sizeof(long));
    *ptr1 = 0x2002;
    // what if alloca 0
    ptr2 = (long *) alloca(0);
    *ptr2 = 0x2003;
}

int main() {
    test();
    return 0;
}

firstly, in assemble code alloca didn't appear to be a function. Instead, it appear with a set of instruction. So, the asm will not have any instruction like call alloca.

Before alloca is called, the stack looks very normal

:> pxr @ rsp
0x7fffe7ddb1d0 0x0000000000001000   ........ @ rsp 4096
0x7fffe7ddb1d8 0x0000000000001001   ........ 4097
0x7fffe7ddb1e0 0x0000000000001002   ........ 4098
0x7fffe7ddb1e8 0x65ba763434aa8f00   ...44v.e
0x7fffe7ddb1f0 0x00007fffe7ddb200   ........ @ rbp [stack] stack R W 0x0

After program execute *ptr = 0x2001. Comparing with previous stack frame, We notice that that stack alloca extra 16 bytes above the initial stack.

:> pxr @ rsp
0x7fffe7ddb1c0 0x0000000000002001   . ...... @ rax 8193
0x7fffe7ddb1c8 0x000055756136632d   -c6auU.. /home/aynakeya/ctf/play/allocaa/a .text sym.__libc_csu_init program ascii ('-') R X 'add rbx, 1' 'a'
0x7fffe7ddb1d0 0x00007fffe7ddb1c0   ........ @ rcx [stack] rsp,rax stack R W 0x2001
0x7fffe7ddb1d8 0x0000000000001001   ........ 4097
0x7fffe7ddb1e0 0x0000000000001002   ........ 4098
0x7fffe7ddb1e8 0x65ba763434aa8f00   ...44v.e
0x7fffe7ddb1f0 0x00007fffe7ddb200   ........ @ rbp [stack] stack R W 0x0

If we continue and execute *ptr = 0x2002. The it will also allocate 16 bytes of memory above the stack. This is because the stack need to be aligned with 0x10. Since we only allocate 8 bytes, there need to be an extra 8 bytes padding in order to make the stack aligned.

Now, lets look what happens if we alloc(0).

After execute *ptr2 = 0x2003. We found the value 0x2002 is replaced by 0x2003. ptr1 and ptr2 are now pointing to a same address. Since

alloc(0) actually return the current rsp address!

:> pxr @ rsp
0x7fffe7ddb1b0 0x0000000000002003   . ...... @ rax 8195
0x7fffe7ddb1b8 0x00007fffe7ddb1e7   ........ [stack] stack R W 0xba763434aa8f0000
0x7fffe7ddb1c0 0x0000000000002001   . ...... @ rsi 8193
0x7fffe7ddb1c8 0x000055756136632d   -c6auU.. /home/aynakeya/ctf/play/allocaa/a .text sym.__libc_csu_init program ascii ('-') R X 'add rbx, 1' 'a'
0x7fffe7ddb1d0 0x00007fffe7ddb1b0   ........ @ rcx [stack] rsp,rax stack R W 0x2003
0x7fffe7ddb1d8 0x00007fffe7ddb1b0   ........ [stack] rsp,rax stack R W 0x2003
0x7fffe7ddb1e0 0x0000000000001002   ........ 4098
0x7fffe7ddb1e8 0x65ba763434aa8f00   ...44v.e
0x7fffe7ddb1f0 0x00007fffe7ddb200   ........ @ rbp [stack] stack R W 0x0

0x3 Some thoughts

it is pretty interesting that there is a function that allocate memory on the stack. It seems very convenient compare to malloc if we just want use a memory space temporaryly. Since it will automatically freed after function exists, we don't need to worry about dangling pointer or memory leak.

But it is also kind of dangerous to use. For example, If we allocate 0 bytes, it would return current rsp address.

Also, when allocate larger space, we may run out of stack address if we use alloca.