Skip to main content
gamedev.net
🔒 Locked

buffer overwrite debugging in VC++

Started by daniel_i_l 14 replies 1.8k views
Original Post
daniel_i_l
daniel_i_l
Hi, i noticed that sometimes one of the variables that I'm printing out gets a value of something like #FD. i think that this means that I'm over writing something and it's flowing into my var - is this right? How do i debug this in VC++? Thanks.
"We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the internet, we know this is not true." -- Professor Robert Silensky
ToohrVyk
ToohrVyk
Set a watch on the variable. It will interrupt execution whenever its value is changed, and pinpoint the code where the change is performed.
Blog — Facebook
daniel_i_l
daniel_i_l
Ok, the debugger said that I got an access violation and stopped at this code from the stl code:
inline void __CLR_OR_THIS_CALL _Container_base::_Orphan_all() const	{	// orphan all iterators	_Lockit _Lock(_LOCK_DEBUG);	if (_Myfirstiter != _IGNORE_MYITERLIST)		{		for (_Iterator_base **_Pnext = (_Iterator_base **)&_Myfirstiter;			*_Pnext != 0; *_Pnext = (*_Pnext)->_Mynextiter)			(*_Pnext)->_Mycont = 0;		*(_Iterator_base **)&_Myfirstiter = 0;		}	}


I did it again and it was here:
		// TEMPLATE FUNCTION _Destroy_rangetemplate<class _Ty,	class _Alloc> inline	void _Destroy_range(_Ty *_First, _Ty *_Last, _Alloc& _Al)	{	// destroy [_First, _Last)	_Destroy_range(_First, _Last, _Al, _Ptr_cat(_First, _Last));	}

what does this mean?
Thanks.
"We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the internet, we know this is not true." -- Professor Robert Silensky
ToohrVyk
ToohrVyk
This means you are using a deallocated variable, which is a bad thing.
Blog — Facebook
daniel_i_l
daniel_i_l
What does that mean? What is "using a deallocated variable"?
Thanks.
"We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the internet, we know this is not true." -- Professor Robert Silensky
ToohrVyk
ToohrVyk
This means that the area of memory that you use does not belong to you. Either you have a pointer that points to somewhere it shouldn't, or you allocated memory, used it, deallocated it and used it again (either with new, or as part of a container).

Of course, there is still the probability that you have a non-deterministic bug in your code that randomly bombs areas of memory through overflows. Did you do that watch thing I told you about?
Blog — Facebook
daniel_i_l
daniel_i_l
Thanks, how do you set a watch on a variable without setting a breakpoint?
Thanks.
"We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the internet, we know this is not true." -- Professor Robert Silensky
ToohrVyk
ToohrVyk
Check your debugger documentation for this — I don't even know what debugger you are using, there's no way I could tell you that.
Blog — Facebook
daniel_i_l
daniel_i_l
the standard VC++ debugger.
Thanks.
"We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the internet, we know this is not true." -- Professor Robert Silensky
ToohrVyk
ToohrVyk
Place a breakpoint when the variable is created, select the variable name in the code, right-click and choose to watch it. Remove the program and resume the running.

You might need to evaluate the address of the variable, and watch the contents at that adress instead.
Blog — Facebook
daniel_i_l
daniel_i_l
Ok, I did the watch variable and found the lines that make the problem, they are:

m_vCollisions.Body1->m_MyState.AngularVelocity += (Impulse/state_a.MomentOfInertia)*(R_ap.Dot(Normal)); m_vCollisions.Body2->m_MyState.AngularVelocity -= (Impulse/state_b.MomentOfInertia)*(R_bp.Dot(Normal));


could that cause a problem?
Thanks.
"We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the internet, we know this is not true." -- Professor Robert Silensky
ToohrVyk
ToohrVyk
Well, the obvious questions are:
  • Is i within permitted index values?
  • Does content of the array at index i have a valid Body1 pointer to an existing object?
    • Blog — Facebook
    daniel_i_l
    daniel_i_l
    -yes
    -yes

    I still can't figure out what the problem is:(
    When it gets to th\t line it doesn't always crash, it seems to crash sort of randomly. What could this mean?
    Thanks.
    "We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the internet, we know this is not true." -- Professor Robert Silensky
    Evil Steve
    Evil Steve
    What version of VC++ are you using? VC2005's STL implementation will break to the debugger if you access an invalid location in the vector, which pre-VC2005 STL implementations don't do.

    It's usually impossible to tell if one line of code is going to cause a problem, it like saying "Have I used to correct version of the word 'there'?" - It's absolutely impossible to tell without some sort of context.
    daniel_i_l
    daniel_i_l
    I'm using Express - 2005. I got this on msdn:
    Quote:

    To use the debug build to check for memory overwrite, you must first rebuild your project for debug. Then, go to the very beginning of your application's InitInstance function and add the following line:

    afxMemDF |= checkAlwaysMemDF;


    What is the InitInstance function and were do I find it?
    Thanks.
    "We've all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the internet, we know this is not true." -- Professor Robert Silensky

    Topic Locked

    This topic has been locked by a moderator. New replies are not allowed.

    Sign in to reply to this topic.