Skip to main content
GameDev.net gamedev.net
🔒 Locked

DX9 C++ Creating Stencil Buffer Silhouette

Started by Buckeye Jul 29, 2010 at 8:19 AM 1 replies 3.1k views
Original Post
Buckeye
Buckeye
Question: Is there a way to set the stencil buffer from a transparent texture or something similar (vs creating a silhouette object/vertexbuffer/etc.)? Is there a technique for setting the stencil better or faster than rendering triangles?

I'm new to playing with the stencil buffer, so I'm sure there are techniques and ways to do things I'm not aware of.

I have a transparent texture for a dashboard and windshield that I mask over the driver's view in a race car game at the end of every frame via a textured quad the size of the backbuffer. It works but is mildly expensive in render time. In addition, of course, it overwrites portions of the color buffer that I've just spent time rendering. At least the stencil would give me an early cull for the pixel.

To save a bit of rendering time, I want to stencil the silhouette of the dashboard/windshield/rearview-mirror. I'm in the process of creating the silhouette object to render for stenciling. It's a vertexbuffer (position only for now), rendered with color and depth turned off. It works but the silhouette isn't as smooth on the curves as I can get in a texture.

Is there such a thing as a "stencil texture?" I.e., a surface of some sort that I can create offline and "blit" to the stencil buffer.

Suggestions?

EDIT: MJP stated back in 2008, "You have to render something, otherwise you can't set any values in the stencil buffer. No way around that." Guess that answers the question about a "stencil texture."

That post did mention using an alpha texture which I'll have to give a try. Still requires a full-buffer quad but sounds like it might fill the bill.

In that same post, S1CA stated: "a pixel that fails alpha test won't be written to (or tested against) the stencil buffer or depth buffer."

In "The Direct3D Graphics Pipeline": If the pixel passes the alpha, depth and stencil tests,.. Is that always the fixed order of testing a pixel? FURTHER READING: Guess that should be rephrased: "Does alpha testing always precede depth/stencil testing?"

Just trying to confirm that alpha testing is the earliest cull for a pixel.


[Edited by - Buckeye on July 29, 2010 10:14:55 AM]
Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly. You don't forget how to play when you grow old; you grow old when you forget how to play.
Demirug
Demirug
The order of these tests in hardware is not exactly defined. They could even be done multiple times on different stages of the pipeline. Most hardware tries to do the depth test as early as possible. Some chips do the same for the stencil test. But if you have the alpha test enabled and write to the depth or stencil buffer things can change.

In any case the hardware/driver tries to reject pixel as soon as it is possible based on the technique that is available.
.
Buckeye
Buckeye
Appreciate the confirmation, Demirug, rather, the lack thereof. It appears that the alpha test may or may not be the earliest cull, but it's certainly a method to convert a texture to a stencil.

Using a couple references for stenciling, I put together something that works. Haven't done extensive performance testing yet but:

1. created an A8 texture for the mask (alpha > 0) that I want (PhotoShop->png->DXTextureTool)
2. enabled alpha testing, CMP_GREATER to ref = 0
3. disabled the color buffer
4. enabled stenciling, CMP_ALWAYS, STENCILPASS=REPLACE with REF = 1
5. render a full-screen (ortho) textured quad (position & tex coords) with the A8 texture
6. for the rest of the scene: re-enabled color, alphafunc = CMP_ALWAYS, STENCILPASS=KEEP

Seems to work. Per your comments, I don't know if this is the earliest cull, but it's what I was looking for, as I can create an alpha texture easier than an object of the same shape for the stencil.

Anyone have a better method?
Please don't PM me with questions. Post them in the forums for everyone's benefit, and I can embarrass myself publicly. You don't forget how to play when you grow old; you grow old when you forget how to play.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.