Skip to main content
gamedev.net
🔒 Locked

Stencil Buffer Corruption

Started by Omaha 5 replies 450+ views
Original Post
Omaha
Omaha
Recently I've been having more trouble with the stencil buffer than I'd like to admit. No matter how I render to it or clear it, it will always be filled with ugly artifacts and junk. Here is a dump of the stencil buffer from a recent run. Also, if I attempt to use the GL_INCR stencil operation, the value in the stencil buffer (if its not one of the junk pixels) will increment from 0 to 1 in the first pass, but will then go from 1 to 0 in the second pass. This makes it seem like I do not really have the 8 bit stencil buffer I requested with my pixelformat (Win32) even though GL_STENCIL_BITS is 8 and my pixel format does have a StencilBits value of 8 as checked via DescribePixelFormat. The junk in the stencil buffer was reproduced on a friend's computer, who has a different video card than me as well on another one of my computers which was using software rendering. The inability to increment has occured on my two computers as well as two friends' computers. Relevent code furnished upon request, but does anyone have some immediate suggestions? [edited by - Omaha on July 31, 2003 12:54:01 PM]
Omaha
Omaha
I''ve tried varying pixel formats, but the artifact problem refuses to go away.
Omaha
Omaha
One final bump for great justice in the hopes someone knows anything about this or a similar problem.
ShlomiSteinberg
ShlomiSteinberg
Never happend to me. Post your rendering code.

"C lets you shoot yourself in the foot rather easily. C++ allows you to reuse the bullet!"
Member of the Shove Pouya Off A Cliff club, SPOAC. (If you wish to join, add this line to your signature.)Member of "Un-Ban nes8bit" association, UNA (to join put this in your sig)
Omaha
Omaha
Here is the stencil buffer management code I''m using for now:



// wrapper function for enabling the stencil test

void EnableStencil()
{
	glEnable(GL_STENCIL_TEST);
}

// ditto for disabling

void DisableStencil()
{
	glDisable(GL_STENCIL_TEST);
}

// all drawing after this call will affect the stencil buffer

// and the depth buffer unless that''s disabled

void BeginStencilDraw()
{
	glStencilMask(GL_TRUE);
	glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
	glStencilOp(GL_KEEP, GL_KEEP, GL_INCR);
}

// return to normal rendering

void EndStencilDraw()
{
	glStencilMask(GL_FALSE);
	glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
	glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
}

// increment global stencil value, so that we only draw on an

// area we just drew in the stencil buffer

void EnterStencil()
{
	if(StencilValue==255)
		return;
		
	glStencilFunc(GL_EQUAL, ++StencilValue, 0xff);
}

// decrement global stencil value to draw in the previous stencil

// value

void ExitStencil()
{
	if(!StencilValue)
		return;
	
	glStencilFunc(GL_EQUAL, --StencilValue, 0xff);
}

// clear the stencil buffer

void ResetStencil()	//	expensive

{
	glStencilMask(GL_TRUE);
	glClear(GL_STENCIL_BUFFER_BIT);
}
theoddbot
theoddbot
Hi,

Try removing:


glStencilMask(GL_TRUE);



GL_TRUE is 1, which is only the Least Significant Bit of the stencil. You clear it straight afterwards, which will only reset the LSB of the stencil to 0, and leave the rest the same.

The default value should be fine, you shouldn''t have to use glStencilMask( ).


cheers,
-Sypro
Omaha
Omaha
The glStencilMask() line is in there actually due to a previous version of the code where if the stencil value was zero the stencil mask would be zeroed out; now I just turn the stencil test on and off.

I misread the documentation for glStencilMask() and thought it was just a straight boolean mask like ColorMask or DepthMask. I had been spending so much time combing over the docs on other stencil functions I overlooked the obvious. Shame on me. I will try that change and see what happens. Thanks a bunch!

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.