Debugging and Understanding Stack Overflows

Another blog post for my readers to enjoy (hopefully), tonight I’m going to explain how stack overflows are caused and how to debug a stack overflow. They are usually always result in a Stop 0x7F with a Double Fault (8) as the first parameter.

Firstly, I think it’s important to understand what is a Stack Overflow and how it’s produced. It’s a common problem when using Recursive functions, that is to say, functions which call themselves. The stack is a area of memory reserved for a process, which is used to store the local variables and function return addresses. A Stack Overflow, typically happens when this area of memory is completely exhausted (used up), and then the function calls return into areas of memory which do not belong to the process. It can also be caused by variables which are too large.


It’s important to understand, that the current stack provided may be corrupt or completely useless, and this will explain the reasoning for the debugger to use the .tss command on the TSS (Task State Segment).

A TSS is a structure used by the Windows Kernel to keep track of Task Management, and any running tasks. The TSS will contain the stack pointers, which is important in relation to this type of problem. I won’t explain completely the internals of TSS.

Anyhow, when a interrupt occurs (exception in this case), the kernel looks at the TSS and loads a stack frame pointer field into the Stack Segment, which enables the Kernel to have a different stack to the program, therefore preserving the stack if anything should go wrong. These stacks used by the TSS are stored in a Interrupt Stack Table, which will allow the Kernel to load the other stack, and allows the Kernel to use good stacks in case of a serious error like the Double Fault we are facing.


References:

Kernel Stacks (x86/x64) 

Task State Segment Wiki 

Modern Memory Management Part 2 

Difference between a Stack Overflow and Segmentation Fault

Debugging a Stack Overflow




 

Advertisements

About 0x14c

I'm a Computer Science student and writer. My primary interests are Graph Theory, Number Theory, Programming Language Theory, Logic and Windows Debugging.
This entry was posted in Debugging. Bookmark the permalink.

One Response to Debugging and Understanding Stack Overflows

  1. VnSpl0it says:

    Great, Thanks !

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s