Skip to main content
gamedev.net
🔒 Locked

Stack overflow?

Started by Fuzztrek Dec 30, 2002 at 9:45 PM 12 replies 650+ views
Original Post
Fuzztrek
Fuzztrek
Hello, This is gonna be pretty vague, because I''ve never experienced a stack overflow before. I have a stack overflow! My program would break after I converted it from dx8 to dx9. I don''t know where to start, if someone could point me in the right direction that would be great
CheeseGrater
CheeseGrater
Hard to say without knowing more about what you're doing but...

Stack overflows are usually caused by running out of stack. The Two biggest culprits for this kind of thing are really big stack based variable declarations - Something like int hugearray[999999999999999999];

The other would be a really deep or infinite recursion, something like


void barf(void) {

barf();

}


[edited by - Cheesegrater on December 30, 2002 11:02:41 PM]
Fuzztrek
Fuzztrek
Edit: Actually it must be somewhere in my MD2 Loading code. I'll have to root through it tomorrow to see if I can figure out what's causing it. It worked fine before I converted it..


[edited by - Fuzztrek on December 30, 2002 11:49:03 PM]
djsteffey
djsteffey
what i would do is create a log file
at the beginning of say each function or each important part of a function...write out to the logfile what you are getting ready to do. After your program fails...open the logfile to see what the last thing written was. Then obviously, the error is in that section.

"I pity the fool, thug, or soul who tries to take over the world, then goes home crying to his momma."
- Mr. T
Andre Luiz Silva
Andre Luiz Silva
do you use recursion in your program? If so, make sure that the stop condition is satisfied. Too many recursions will make you run out of stack space.

"- To begin with, said the Cat, a dog''s not mad. You grant that?
- I suppose so, said Alice.
- Well, then, - the Cat went on - you see, a dog growls when it''s angry, and wags its tail when it''s pleased. Now I growl when I''m pleased, and wag my tail when I''m angry. Therefore I''m mad."
"- To begin with, said the Cat, a dog's not mad. You grant that? - I suppose so, said Alice. - Well, then, - the Cat went on - you see, a dog growls when it's angry, and wags its tail when it's pleased. Now I growl when I'm pleased, and wag my tail when I'm angry. Therefore I'm mad."
Fuzztrek
Fuzztrek
Okay, I''ll try that.

I don''t think I have any recursion in my program (in fact I''m pretty sure I don''t). I''ll try adding to my log file and see what comes out. Thanks!!
Fuzztrek
Fuzztrek
I tried it, and found out that it's failing on D3DXFillTexture in a PCX file loader in my engine. I didn't write the loader (gasp) so I'm not exactly sure how to use this function. I've figured out pretty much what it does and how it works, however if someone could tell me what's changed or what has happend that is causing this error that would be great!

Here is the whole function (the D3DXFillTexture is at the bottom of the code)


      
HRESULT LoadPcx(LPCTSTR FileName, IDirect3DDevice9 *pd3ddev, BOOL MipMap, _D3DFORMAT Format, IDirect3DTexture9 **ppTex)
{
CFileHandle hFile = CreateFile(FileName, GENERIC_READ, FILE_SHARE_READ, 0, OPEN_EXISTING, 0, 0);
if (hFile == INVALID_HANDLE_VALUE) {
ATLTRACE(_T("LS_PCXLDR: Cannot open file: %s\n"), FileName);
return HRESULT_FROM_WIN32(GetLastError());
}

CKernelHandle hFileMapping = CreateFileMapping(hFile, 0, PAGE_READONLY, 0, 0, 0);
if (!hFileMapping) {
ATLTRACE(_T("LS_PCXLDR: Cannot create file mapping for: %s\n"), FileName);
return HRESULT_FROM_WIN32(GetLastError());
}

CViewOfFilePointer pFileView = MapViewOfFile(hFileMapping, FILE_MAP_READ, 0, 0, 0);
if (!pFileView) {
ATLTRACE(_T("LS_PCXLDR: Cannot map view of file for: %s\n"), FileName);
return HRESULT_FROM_WIN32(GetLastError());
}

PCX_HEADER *pHeader = (PCX_HEADER *) (PVOID) pFileView;
if (pHeader->Manufacturer != 10) {
ATLTRACE(_T("LS_PCXLDR: Wrong manufacturer: %s\n"), FileName);
return HRESULT_FROM_WIN32(ERROR_BAD_FORMAT);
}

if (pHeader->Encoding != 1) {
ATLTRACE(_T("LS_PCXLDR: %s: Only RLE encoding is supported\n"), FileName);
return HRESULT_FROM_WIN32(ERROR_BAD_FORMAT);
}

if (pHeader->Bpp != 8) {
ATLTRACE(_T("LS_PCXLDR: %s: Only 8-bpp PCXs are supported\n"), FileName);
return HRESULT_FROM_WIN32(ERROR_BAD_FORMAT);
}

if (pHeader->NPlanes != 1 && pHeader->NPlanes != 3) {
ATLTRACE(_T("LS_PCXLDR: %s: Only 1-plane 8-bit and 3-plane 24-bit pcxs are supported\n"), FileName);
return HRESULT_FROM_WIN32(ERROR_BAD_FORMAT);
}

UINT Width = pHeader->WindowXmax-pHeader->WindowXmin+1;
UINT Height = pHeader->WindowYmax-pHeader->WindowYmin+1;

HRESULT hr = D3DXCreateTexture(pd3ddev, Width, Height, MipMap ? 0 : 1, 0, Format, D3DPOOL_MANAGED, ppTex);
if (FAILED(hr)) {
ATLTRACE(_T("LS_PCXLDR: %s: Cannot create texture\n"));
return hr;
}


D3DSURFACE_DESC desc;
(*ppTex)->GetLevelDesc(0, &desc);
// if (desc.Width != Width || desc.Height != Height) {

// ATLTRACE(_T("LS_PCXLDR: %s: Texture was created with improper dimensions\n"));

// return D3DERR_NOTAVAILABLE;

// }


FILL_TEXTURE_DATA ftd = {
Width, Height, new DWORD[Width*Height],
};

if (pHeader->NPlanes == 1) {
// Read palette

DWORD FileSize = GetFileSize(hFile, 0);
BYTE *PaletteMagic = AdvanceCastPtr<BYTE>((PVOID) pFileView, FileSize-769);

if (*PaletteMagic != 12) {
ATLTRACE(_T("LS_PCXLDR: %s: Bad palette information in 8-bit pcx file\n"));
delete ftd.pLoadedData;
return HRESULT_FROM_WIN32(ERROR_BAD_FORMAT);
}

RGBTRIPLE *pSourcePalette = (RGBTRIPLE *) (PaletteMagic+1);
RGBQUAD *pPalette = new RGBQUAD[256];
RGBQUAD *pDestPalette = pPalette;

for (int i = 0; i < 256; ++i) {
pDestPalette->rgbBlue = pSourcePalette->rgbtRed;
pDestPalette->rgbGreen = pSourcePalette->rgbtGreen;
pDestPalette->rgbRed = pSourcePalette->rgbtBlue;
pDestPalette->rgbReserved = 0;
++pDestPalette;
++pSourcePalette;
}

Decode8bitPcx(pHeader, Width, Height, (RGBQUAD *) ftd.pLoadedData, pPalette);

delete pPalette;
} else {
Decode24bitPcx(pHeader, Width, Height, ftd.pLoadedData);
}

#if 0
ATLASSERT(Format == D3DFMT_X8R8G8B8 || Format == D3DFMT_A8R8G8B8);
D3DLOCKED_RECT lr;
pTex->LockRect(0, &lr, 0, 0);

for (UINT i = 0; i < Height; ++i)
{
memcpy(AdvancePtr(lr.pBits, i*lr.Pitch), ftd.pLoadedData+i*Width, Width*4);
}

(*ppTex)->UnlockRect(0);
#else

D3DXFillTexture((*ppTex), (LPD3DXFILL2D) FillTextureProc, &ftd);

#endif

delete ftd.pLoadedData;
return S_OK;
}


Heh. I so feel like a newbie right now.. posted all this code and expected someone else to find the error >.<

Edit:

Ah! run in debug mode, it sends me here (I forgot to include this in the code above):


  
void __cdecl FillTextureProc(D3DXVECTOR4 *pOut, D3DXVECTOR2 *pTexCoord, D3DXVECTOR2 *pTexelSize, FILL_TEXTURE_DATA *pftd) {
ATLASSERT(pTexCoord->x < 0.9999 && pTexCoord->y < 0.9999);
UINT x = UINT(pTexCoord->x*pftd->Width); // Problem line

UINT y = UINT(pTexCoord->y*pftd->Height);
DWORD *pSourceImage = pftd->pLoadedData+x+y*pftd->Width;
RGBQUAD *prgbq = (RGBQUAD *) (pSourceImage);
if (*pSourceImage) {
int x = 2;
}
pOut->x = prgbq->rgbRed/255.f;
pOut->y = prgbq->rgbGreen/255.f;
pOut->z = prgbq->rgbBlue/255.f;
pOut->w = 0;
}


[edited by - Fuzztrek on December 31, 2002 4:53:22 PM]
Thrump
Thrump
Ah, stack overflows. Haven''t seen one of those in a LONG time.

Some stuff to try if all else fails:

Find out how much memory windows gives you for the stack.
Find out how much of that you''re using at key points of the program.
Try and increase the amount windows gives you. (not a solution, but may give you some more information)

It may be that your program was very close to exhausting the stack, and the move to DX9 was the straw the broke the camels stack... err, back.
Fuzztrek
Fuzztrek
Oh dear. Do you ever get that feeling as though you feel you''ve done something that should never ever be done? Something thats slowing down your program and eating up resources, and you don''t know why? :| Thats how I feel.
Fuzztrek
Fuzztrek
Bump. Anyone know how I could find out how much memory windows is giving me?
AikonIV
AikonIV
OK, I did a forum search on stack overflows and came up with this thread. I too am having a stack overflow for unknown reasons. It seems to be window's CreateDialog that's doing it for me tho...

So, BUMP I guess. How can I find out how much stack windows is giving me? And how can I find out how much I am using?

[edited by - AikonIV on March 5, 2003 1:51:41 PM]
LessBread
LessBread
Rather than reviving a dead thread - albeit a newly dead thread - you should probably start a completely new thread to ask your question.

That said, on Windows the default stack size is 1 Mb. As for tracking stack usage, I have a couple of ideas, but nothing certain, so I''ll skip that question.

I find it unlikely that CreateDialog would produce a stack overflow.
"I thought what I'd do was, I'd pretend I was one of those deaf-mutes." - the Laughing Man
AikonIV
AikonIV
Oops, my bad.

Yes, I probably should''ve made a new thread, but oh well. It turns out that I forgot everything I knew about modeless dialog message functions and was calling DefDlgProc at the end of it. This caused infinite recursion eventually overflowing the stack.
Jan Wassenberg
Jan Wassenberg
Call stack is your friend
E8 17 00 42 CE DC D2 DC E4 EA C4 40 CA DA C2 D8 CC 40 CA D0 E8 40E0 CA CA 96 5B B0 16 50 D7 D4 02 B2 02 86 E2 CD 21 58 48 79 F2 C3

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.