Skip to main content
gamedev.net
🔒 Locked

Stencil Shadow problem (solved)

Started by azjerei Jan 14, 2005 at 12:11 PM 4 replies 900+ views
Original Post
azjerei
azjerei
I am having major problems with my stencil shadows. They turn out white!!! Here is an image: Here is the code I am using to render the scene and calculating the shadow:

//---------------------------------------------------------------
//
//	void RenderSceneWithShadows()
//
//	Render the world geometry, with stencil shadows.
//
//---------------------------------------------------------------
void RenderSceneWithShadows(int lightingFlicker)
{
	// Disable writing to the depth and color buffers, we only
	// want to write to the stencil buffer.
	glColorMask(GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE);
	glDepthMask(GL_FALSE);

	// Enable stencil buffer.
	glEnable(GL_STENCIL_TEST);

	// Set the stencil buffer to 1.
	glStencilFunc(GL_ALWAYS, 1, 0xFFFFFFFF);
	glStencilOp(GL_REPLACE, GL_REPLACE, GL_REPLACE);

	// Render the level geometry.
	RenderLevel(0, false);

	// Turn on color and depth buffers.
	glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
	glDepthMask(GL_TRUE);

	// Only write to the stencil buffer (value is 1) and keep the buffer content.
	glStencilFunc(GL_EQUAL, 1, 0xFFFFFFFF);
	glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);

	// Render level geometry in the second pass.
	RenderLevel(lightingFlicker, true);

	// Push on a new matrix.
	glPushMatrix();

	glColor4f(0.0f, 0.0f, 0.0f, 0.5f);

	// Disable texturing, lighting and depth test, and enable blending.
	glDisable(GL_TEXTURE_2D);
	glDisable(GL_LIGHTING);
	glDisable(GL_DEPTH_TEST);
	glEnable(GL_BLEND);

	glStencilOp(GL_KEEP, GL_KEEP, GL_INCR);

	float groundplane[4] = {0.0, 1.0f, 0.0, 1.0f};

	// Calculate the projected shadow.
	shadowmatrix(floorShadow, groundplane, SunPosition);
	glMultMatrixf((float *) floorShadow);

	// Render player.
	RenderPlayer(0, false);

	// Re-enable texturing, lighting and depth test, and disable blending.
	glEnable(GL_TEXTURE_2D);
	glEnable(GL_LIGHTING);
	glEnable(GL_DEPTH_TEST);
	glDisable(GL_BLEND);

	// Pop the matrix.
	glPopMatrix();

	// Create a new matrix.
	glPushMatrix();

	// Disable the stencil buffer.
	glDisable(GL_STENCIL_TEST);

	// Render player model.
	RenderPlayer(lightingFlicker, true);

	// Pop the matrix (never liked those movies anyways :P).
	glPopMatrix();
}


The RenderLevel(..) and RenderPlayer(..) functions just render the world geometry and the player model, nothing special is done there. Though, I have noticed that whenever I change the Clear Color to something, the shadow changes color as well. And if I set it to black, I don't see the shadow at all (i.e. as with blending.. black is turned completely transparent). So, something is definately wrong. If you need to look at other pieces of the code, let me know. [Edited by - azjerei on January 15, 2005 3:45:18 AM]
kburkhart84
kburkhart84
I actually use this same method to draw shadows. Maybe you have done this in the RenderPlayer function, but you need to turn off textures and lighting and make the drawing color the shadows color. Also, make the alpha blend with the texture of the floor/wall. I do it this way and it works fine.


azjerei
azjerei
I have turned off everything when rendering both the geometry and the player model for the first pass (lighting calculations, texturing, collision detection), but the second time it's all there.

With the "make the drawing color the shadows color", do you mean that I should define a glColor4f(0, 0, 0, 1.0f) somewhere (since it's supposed to be black)? If so, where would you recommend? I have it in that code there already.. but it does not do anything.

And with the alpha blending... where and what should I put? I have tried all of the glBlendFunc(...) definitions that I know, but none did the trick (one made the shadow just a translucent of the original model... which looked really weird).
kburkhart84
kburkhart84
Do it like this.
1. Draw floor into stencil buffer.
2. Multmatrix like you already do.
3. No textures glColor3f(0,0,0) No lighting
4. Draw shadow model using the stencil
5. Turn back on Textures lighting glColor3f(1,1,1)
6. Draw objects normal
When I calculate the matrix for the shadow, I use the plane of the floor, but I raise it 0.1 up on the normal(the D in the plane equation) that way it is above the floor and I don't have to do anything with the depth buffer. The blending part shouldn't be too hard to implement after these steps.


azjerei
azjerei
If I wanted some really good shadows, I should not stick with a static normal as my geometry is very curved with hills and stuff, though I have absolutely no idea how to take each vertex normal into consideration when I calculate the projection matrix.

I'll try what you have written, thank you.
azjerei
azjerei
Got it! I defined glBlendFunc(GL_ZERO, GL_ONE_MINUS_SRC_COLOR); just before calculating the shadow matrix and rendering the shadow model.

Thank you both! :)

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.