if (m_bNotInMenu)
{
g_userInterface.render();
}
While testing last night, all of a sudden the entire UI stopped being rendered. I did a search for anywhere that the m_bNotInMenu can become true, and there is no way that that single line of code can ever be reached while within the code where the UI disappeared. So now I'm fearing the worst... memory overwrite? Has anyone else experienced something similar in their programming career? What is your best intuition? -dbh
🔒 Locked
Strange Memory Issues
Advertisement
Original Post
I *almost* released the game last night, but I encountered one last series of bugs that completely bewilders me. When I do a release build and play through the game, I am almost certain that certain variables are being overwritten. For example, in my code I have a simple boolean that blocks the entire UI from being drawn while in the menu (the game UI, that is):
That's exactly what I just did, actually... good call :)
While debugging in release mode, I put a break point next to all the places in the entire game where that variable is set to true. I play the game, and after about 4 minutes of playing, the UI disappears and the variable has been set-- but none of my break points were hit.
So yeah, that's a memory overwrite... right? :D
I am fairly certain it is somewhere in the code for enemy leaks... it takes a lot of them to leak (more than 5 almost always, but less than 10)-- but eventually that bool is flipped when an enemy leaks.
I checked the code for deleting dead enemies, and it looks just fine and seems to operate perfectly.
-dbh
While debugging in release mode, I put a break point next to all the places in the entire game where that variable is set to true. I play the game, and after about 4 minutes of playing, the UI disappears and the variable has been set-- but none of my break points were hit.
So yeah, that's a memory overwrite... right? :D
I am fairly certain it is somewhere in the code for enemy leaks... it takes a lot of them to leak (more than 5 almost always, but less than 10)-- but eventually that bool is flipped when an enemy leaks.
I checked the code for deleting dead enemies, and it looks just fine and seems to operate perfectly.
-dbh
Oh, I also disabled optimizations for release builds, and it is still happening. So I'd wager it also happens in debug mode-- I don't think thats the problem. I think it's something in my code somewhere :|
I've never actually dealt with an overwrite before though, so I was just wondering what the general strategy for tracking these down was.
Thanks!
-dbh
I've never actually dealt with an overwrite before though, so I was just wondering what the general strategy for tracking these down was.
Thanks!
-dbh
Rattrap, I'll try that; that's actually a really good idea.
Sneftel, how do you do that? I was wondering if that was possible, but I've never played with direct memory access like that in Visual Studio .NET 2003.
I have several classes to run to, as well as work, so I'll try these suggestions and let you know how it goes later this evening. Thanks again!
-dbh
Sneftel, how do you do that? I was wondering if that was possible, but I've never played with direct memory access like that in Visual Studio .NET 2003.
I have several classes to run to, as well as work, so I'll try these suggestions and let you know how it goes later this evening. Thanks again!
-dbh
When you insert a new breakpoint (ms.net 2003) you have
a few tabpages (function/file/adress/data).
Put a breakpoint somewhere where on the line where you initialize
your variable and then insert a new one, data and enter the name
of your variable, 1 elements.
Then you should have a breakpoint on a memory-adress, which you
can confirm that it's working by just resuming execution of your
program - and it should halt immediately because you are initializing
a value to your variable (thus modifying the memory location).
In vc6 I always used to but the &myvar in the watch-window, so
you could see the actual adress of the var - then when you
inserted the breakpoint you had to use this adress.
Regards
a few tabpages (function/file/adress/data).
Put a breakpoint somewhere where on the line where you initialize
your variable and then insert a new one, data and enter the name
of your variable, 1 elements.
Then you should have a breakpoint on a memory-adress, which you
can confirm that it's working by just resuming execution of your
program - and it should halt immediately because you are initializing
a value to your variable (thus modifying the memory location).
In vc6 I always used to but the &myvar in the watch-window, so
you could see the actual adress of the var - then when you
inserted the breakpoint you had to use this adress.
Regards
It's a bit of a pain in the butt until .NET 2005, IMHO, but the basic idea in 2003 is:
1. Break somewhere where the variable in question is accessible.
2. Put a watch on the variable's address, and copy down that address.
3. Set a new "data breakpoint" (Debug->New Breakpoint..., select "Data" tab) with the variable "*(int*)0x12345678" (change "int" to the appropriate datatype, and the address to the address you just found) and clear the "context" field.
4. Continue execution.
BTW, for member variables, make sure that you aren't actually looking at a different instance of the class.
1. Break somewhere where the variable in question is accessible.
2. Put a watch on the variable's address, and copy down that address.
3. Set a new "data breakpoint" (Debug->New Breakpoint..., select "Data" tab) with the variable "*(int*)0x12345678" (change "int" to the appropriate datatype, and the address to the address you just found) and clear the "context" field.
4. Continue execution.
BTW, for member variables, make sure that you aren't actually looking at a different instance of the class.
Wow, that did the trick. I found a small array I was using to track the number and type of enemy leaks, but I had forgotten that should *only* be done in a certain game type, not one of the variations-- thus it was (as Rattrap suggested) accessing memory beyond the array :|
Thanks again Kitt3n and Sneftel, I feel super powerful with my new breakpoint skill :D
-dbh
Thanks again Kitt3n and Sneftel, I feel super powerful with my new breakpoint skill :D
-dbh
Quote:
It's a bit of a pain in the butt until .NET 2005, IMHO, but the basic idea in 2003 is:
Yeah. If anyone's interested, this can be done directly in code. You write your address and some flags to Dr0..3 (whichever is free) and mark it as active in Dr7. Slight wrinkle: this must be done with SetThreadContext while the target thread is suspended (because debug regs are only accessible from Ring 0). It runs to ~250 LOC.
Quote:
Original post by Jan Wassenberg Quote:
It's a bit of a pain in the butt until .NET 2005, IMHO, but the basic idea in 2003 is:
Yeah. If anyone's interested, this can be done directly in code. You write your address and some flags to Dr0..3 (whichever is free) and mark it as active in Dr7. Slight wrinkle: this must be done with SetThreadContext while the target thread is suspended (because debug regs are only accessible from Ring 0). It runs to ~250 LOC.
Care to expand on that slightly and/or post those 250 lines? Sounds interesting, but having not much messed around with inline assembly past the point of arbitrary breakpoints (hurray for int 3h!), I really don't understand exactly how plugging things into the debugging registers would help anything, no knowing the specific significance of each [grin]
Quote:
Original post by dbh
So now I'm fearing the worst... memory overwrite? Has anyone else experienced something similar in their programming career? What is your best intuition?
-dbh
A friend of me just posted me this link: hardware breakpoints. To explain it a little: the CBreakpoint class is used to setup breakpoints that will be fired when a value is changed. I'm pretty sure you'll find this rather cool :)
Reagrds,
Quote:
Care to expand on that slightly and/or post those 250 lines? Sounds interesting, but having not much messed around with inline assembly past the point of arbitrary breakpoints (hurray for int 3h!), I really don't understand exactly how plugging things into the debugging registers would help anything, no knowing the specific significance of each
Sure, code is below (license = GPL). Debug registers are explained in IA32 Volume 3 (System Programming).
Quote:
A friend of me just posted me this link: hardware breakpoints. To explain it a little: the CBreakpoint class is used to setup breakpoints that will be fired when a value is changed. I'm pretty sure you'll find this rather cool :)
Yep, that's the same principle. FYI that code is a little bit more simple than it should be - you aren't allowed to Get/SetThreadContext on the current thread. That's a reasonable requirement, explicitly mentioned in the docs and indeed causes problems on my WinXP SP2 system.
Quote:
Looked cool until I read the part about being intel specific :(
Well of course it's CPU architecture specific (IA32, to which AMD64 is backwards compatible), but that's all.
//-----------------------------------------------------------------------------// to avoid deadlock, be VERY CAREFUL to avoid anything that may block,// including locks taken by the OS (e.g. malloc, GetProcAddress).typedef int(*WhileSuspendedFunc)(HANDLE hThread, void* user_arg);struct WhileSuspendedParam{ HANDLE hThread; WhileSuspendedFunc func; void* user_arg;};static void* while_suspended_thread_func(void* user_arg){ debug_set_thread_name("suspender"); DWORD err; WhileSuspendedParam* param = (WhileSuspendedParam*)user_arg; err = SuspendThread(param->hThread); // abort, since GetThreadContext only works if the target is suspended. if(err == (DWORD)-1) { debug_warn("while_suspended_thread_func: SuspendThread failed"); return (void*)(intptr_t)-1; } // target is now guaranteed to be suspended, // since the Windows counter never goes negative. int ret = param->func(param->hThread, param->user_arg); err = ResumeThread(param->hThread); debug_assert(err != 0); return (void*)(intptr_t)ret;}static int call_while_suspended(WhileSuspendedFunc func, void* user_arg){ int err; // we need a real HANDLE to the target thread for use with // Suspend|ResumeThread and GetThreadContext. // alternative: DuplicateHandle on the current thread pseudo-HANDLE. // this way is a bit more obvious/simple. const DWORD access = THREAD_GET_CONTEXT|THREAD_SET_CONTEXT|THREAD_SUSPEND_RESUME; HANDLE hThread = OpenThread(access, FALSE, GetCurrentThreadId()); if(hThread == INVALID_HANDLE_VALUE) { debug_warn("OpenThread failed"); return -1; } WhileSuspendedParam param = { hThread, func, user_arg }; pthread_t thread; WARN_ERR(pthread_create(&thread, 0, while_suspended_thread_func, ¶m)); void* ret; err = pthread_join(thread, &ret); debug_assert(err == 0 && ret == 0); return (int)(intptr_t)ret;}////////////////////////////////////////////////////////////////////////////////// breakpoints////////////////////////////////////////////////////////////////////////////////// breakpoints are set by storing the address of interest in a// debug register and marking it 'enabled'.//// the first problem is, they are only accessible from Ring0;// we get around this by updating their values via SetThreadContext.// that in turn requires we suspend the current thread,// spawn a helper to change the registers, and resume.// parameter passing to helper thread. currently static storage,// but the struct simplifies switching to a queue later.static struct BreakInfo{ uintptr_t addr; DbgBreakType type; // determines what brk_thread_func will do. // set/reset by debug_remove_all_breaks. bool want_all_disabled;}brk_info;// Local Enable bits of all registers we enabled (used when restoring all).static DWORD brk_all_local_enables;// IA32 limit; will need to update brk_enable_in_ctx if this changes.static const uint MAX_BREAKPOINTS = 4;// remove all breakpoints enabled by debug_set_break from <context>.// called while target is suspended.static int brk_disable_all_in_ctx(BreakInfo* UNUSED(bi), CONTEXT* context){ context->Dr7 &= ~brk_all_local_enables; return 0;}// find a free register, set type according to <bi> and// mark it as enabled in <context>.// called while target is suspended.static int brk_enable_in_ctx(BreakInfo* bi, CONTEXT* context){ uint reg; // index (0..3) of first free reg uint LE; // local enable bit for <reg> // find free debug register. for(reg = 0; reg < MAX_BREAKPOINTS; reg++) { LE = BIT(reg*2); // .. this one is currently not in use. if((context->Dr7 & LE) == 0) goto have_reg; } debug_warn("brk_enable_in_ctx: no register available"); return ERR_LIMIT;have_reg: // store breakpoint address in debug register. DWORD addr = (DWORD)bi->addr; // .. note: treating Dr0..Dr3 as an array is unsafe due to // possible struct member padding. switch(reg) { case 0: context->Dr0 = addr; break; case 1: context->Dr1 = addr; break; case 2: context->Dr2 = addr; break; case 3: context->Dr3 = addr; break; default: UNREACHABLE; } // choose breakpoint settings: // .. type uint rw = 0; switch(bi->type) { case DBG_BREAK_CODE: rw = 0; break; case DBG_BREAK_DATA: rw = 1; break; case DBG_BREAK_DATA_WRITE: rw = 3; break; default: debug_warn("brk_enable_in_ctx: invalid type"); } // .. length (determine from addr's alignment). // note: IA-32 requires len=0 for code breakpoints. uint len = 0; if(bi->type != DBG_BREAK_CODE) { const uint alignment = (uint)(bi->addr % 4); // assume 2 byte range if(alignment == 2) len = 1; // assume 4 byte range else if(alignment == 0) len = 3; // else: 1 byte range; len already set to 0 } // update Debug Control register const uint shift = (16 + reg*4); // .. clear previous contents of this reg's field // (in case the previous user didn't do so on disabling). context->Dr7 &= ~(0xFu << shift); // .. write settings context->Dr7 |= ((len << 2)|rw) << shift; // .. mark as enabled context->Dr7 |= LE; brk_all_local_enables |= LE; return 0;}// carry out the request stored in the BreakInfo* parameter.// called while target is suspended.static int brk_do_request(HANDLE hThread, void* arg){ int ret; BreakInfo* bi = (BreakInfo*)arg; CONTEXT context; context.ContextFlags = CONTEXT_DEBUG_REGISTERS; if(!GetThreadContext(hThread, &context)) { debug_warn("brk_do_request: GetThreadContext failed"); goto fail; }#if CPU_IA32 if(bi->want_all_disabled) ret = brk_disable_all_in_ctx(bi, &context); else ret = brk_enable_in_ctx (bi, &context);#else#error "port"#endif if(!SetThreadContext(hThread, &context)) { debug_warn("brk_do_request: SetThreadContext failed"); goto fail; } return 0;fail: return -1;}// arrange for a debug exception to be raised when <addr> is accessed// according to <type>.// for simplicity, the length (range of bytes to be checked) is// derived from addr's alignment, and is typically 1 machine word.// breakpoints are a limited resource (4 on IA-32); abort and// return ERR_LIMIT if none are available.int debug_set_break(void* p, DbgBreakType type){ lock(); brk_info.addr = (uintptr_t)p; brk_info.type = type; int ret = call_while_suspended(brk_do_request, &brk_info); unlock(); return ret;}// remove all breakpoints that were set by debug_set_break.// important, since these are a limited resource.int debug_remove_all_breaks(){ lock(); brk_info.want_all_disabled = true; int ret = call_while_suspended(brk_do_request, &brk_info); brk_info.want_all_disabled = false; unlock(); return ret;}HTH+HAND
Topic Locked
This topic has been locked by a moderator. New replies are not allowed.
Sign in to reply to this topic.
Advertisement