Skip to main content
gamedev.net
🔒 Locked

Detecting missing stack frames

Started by ApochPiQ Oct 17, 2008 at 12:35 PM 5 replies 2.4k views
Original Post
ApochPiQ
ApochPiQ
I'm currently using the following code (adapted from a Gamasutra article) to obtain stack frame information on a running EXE: 
unsigned GetCallStackPtr(unsigned numskips)
{
	struct StackFrame
	{
		StackFrame* Previous;
		unsigned ReturnAddress;
	};

	StackFrame* frame;
	__asm mov [frame], ebp;
	
	while (numskips > 0) {
		if (frame->Previous) {
			frame = frame->Previous;
		}
		else {
			return NULL;
		}
		--numskips;
	}

	return frame->ReturnAddress;
}
This works great provided that everything has valid stack frames. The problem is, I link to a third party DLL which isn't built with stack frame information embedded. This DLL has callbacks into my code, which in turn call GetCallStackPtr(). The problem is, as the code walks up the list of calls, eventually it hits that DLL and the stack frame becomes gibberish. At this point, GetCallStackPtr() crashes because it starts dereferencing garbage memory addresses. Does anyone know of a way to detect that a stack frame has gone bogus? Visual Studio and WinDbg can both do it, and IIRC OllyDbg can as well, so I know it's at least theoretically possible... I just haven't got the first clue how to go about doing it.
SiCrane
SiCrane
StackWalk64() should be able to detect bogus stack frames.
ApochPiQ
ApochPiQ
Yeah, only one problem with that - we're harvesting call stack information across several threads, and realtime performance is critical. Therefore, suspending a thread to grab its context and call StackWalk64() is prohibitively costly. (In my past experience, walking the stack that way has had a fairly significant overhead.)

Also, throwing an SEH exception to grab the calling thread's context is pretty costly as well. So both our methods for calling StackWalk64() are more or less not a very good option.
SiCrane
SiCrane
What OS(s) are you targeting? On XP and later you can use RtlCaptureContext() to get a CONTEXT. Pre-Vista, this also generally works to fill a CONTEXT's Eip:
    __asm    call x    __asm x: pop eax    __asm    mov ctx.Eip, eax

But on Vista it seems this can trigger some security safeguards.

I can't really help you with the cost of calling StackWalk64(). AFAIK, all the information required to be filled in the CONTEXT for StackWalk64() are necessary to detect a bogus stack frame, so I don't know if anything you try to do it manually will come in at a significantly cheaper cost than StackWalk64() and still be robust. Though for a very rough validation you might want to just check that the frame->Previous in addition to being non-NULL is also an address higher on the stack than the current frame and is also between the base pointer and the base of the stack. This might be good enough for that particular DLL you're dealing with, but I wouldn't call it a general case solution.
Nypyren
Nypyren
If you make the assumption that data pushed onto the stack is aligned to 32-bit multiples, you can rapidly search the entire stack for 32-bit values that fall within the areas of memory that are marked for execute (use VirtualQuery or GetModuleInfo to scan for execute pages).

Doing a scan like this will find ALL of the return addresses that were pushed onto the stack. The valid area of executable pages is incredibly small compared to the total possible address space, so random data that just SEEMS to be a code pointer will be extremely rare. A wise programmer will be able to ignore those, anyway. The important part is that it won't miss anything.

You could modify the code so that it also checks for EBP pointers to more quickly skip irrelevant stuff.

In the rare case when you have a noncontiguous stack, you hopefully have an EBP chain anyway.

[Edited by - Nypyren on October 17, 2008 4:25:58 PM]
Anon Mike
Anon Mike
This is hard in general because for a while it was fashionable to build your app with frame pointer omission. i.e. don't save ebp on the stack and instead just use it as a general register. People are now realizing that the miniscule improvement you generally get doesn't make up for ruining stack traces and in general making debugging that much harder.

Things can also get fouled up by return value optimization. e.g. if you end your function with "return foo()" then the compiler can fixup the stack and issue a jmp to foo() instead of a call. foo() will the return directly back to your caller and all evidence that you were ever on the stack disappears.

VC/WinDbg use info in the .pdb file to figure out how frames are chained together. If you're seeing them work on random third party stuff you don't have .pdb's for then I don't know.
-Mike
ApochPiQ
ApochPiQ
Well, I've been through several options here:

  • StackWalk64()

  • Checking executability of the return address via VirtualQuery

  • Checking if the return address points to code using SymGetModuleInfo

  • Checking if the previous frame pointer was valid using IsBadReadPtr

  • Checking if the previous frame pointer was valid using VirtualQuery



All of these performed very, very poorly, so they are not realistic options. However, I did have some luck with SiCrane's suggestion of looking at whether or not the memory lies on the stack:

bool IsMemoryOnStack(void* memory){	void* stacktop = 0;	void* stackbase = 0;	__asm	{		mov EAX, FS:[4]		mov stacktop, eax		mov EAX, FS:[8]		mov stackbase, eax	}	return (memory <= stacktop && memory >= stackbase);}


Adding a call to IsMemoryOnStack() inside GetCallStackPtr() has at least fixed the issue for the libraries we currently use, so it looks like it will be robust enough for development purposes (this is never intended to be release-ready code in the first place - the whole thing is a rather large hack for debugging only).


So that's settled that question. Thanks all!

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.