Memory Management/Stacks and Heaps
In most native executable programs, there are two types of memory available: stack-based and heap-based memory.
The system stack, for those systems that have them, are used most often to provide frames. A frame is a way to localize information about subroutines. In general, a subroutine must have in its frame the return address (where to jump back to when the subroutine completes), the function's input parameters. When a subroutine is called, all this information is pushed onto the stack in a specific order. When the function returns, all these values on the stack are popped back off, reclaimed to the system for later use with a different function call. In addition, subroutines can also use the stack as storage for local variables.
This whole dance, the process of creating frames on the stack and reclaiming them happens mostly transparently, and is handled by the compiler. The programmer is typically not aware of this process, and doesn't need to specifically account for it.
One pitfall to worry about can occur when a local array is allocated on the stack. If the function writes to a memory location beyond the end of the array, a problem called a stack overflow occurs. A stack overflow is a problem because data values are stored tightly on the stack and in an overflow you can accidentally overwrite these values. As was mentioned before, the function's return value is stored on the stack. So if this value is overwritten, the function will not return to the correct location. A diligent hacker can use this fact to overflow the stack with very specific values, such as the address of virus code in memory.
Stack storage is fixed. That is, the space allocated on the stack is set at compile time by the compiler and is not altered while the program executes.
The heap is an area of dynamically-allocated memory that is managed automatically by the operating system or the memory manager library. Memory on the heap is allocated, deallocated, and resized regularly during program execution, and this can lead to a problem called fragmentation. Fragmentation occurs when memory objects are allocated with small spaces in between that are too small to hold additional memory objects. The net result is a percentage of the heap space that is not usable for further memory allocations.
Heaps are also susceptible to overflow situations, although the results are typically not as dire as in a stack overflow. It's not impossible for a hacker to disrupt the system through a heap overflow, but it's particularly difficult.