Skip to main content
gamedev.net
🔒 Locked

Stencil Buffer issues....

Started by oxygen728 Nov 16, 2004 at 2:56 AM 7 replies 1.8k views
Original Post
oxygen728
oxygen728
Well... I'm giving shadows my first shot... Currently I am forced to backtrack to see what the problem is... and I'm all the way back to this line of code: g_pd3dDevice->Clear( 0, NULL, D3DCLEAR_TARGET|D3DCLEAR_ZBUFFER|D3DCLEAR_STENCIL, D3DCOLOR_XRGB(0,0,255), 1.0f, 0 ); Simply clearing the stencil buffer makes things go wacko. I can turn off all shadowing code of any kind, and if I clear my stencil buffer, my scene is drawn crazy - Bits and pieces of things are drawn at times, while my skybox is drawn in full just fine at all times! (I draw it within 1 unit of the camera) If I comment out the part that clears the stencil buffer... the scene works fine... I just can't figure out why this is happening. Thanks! Brian
alf259
alf259
IDirect3DDevice9::Clear will fail if you:

a) Try to clear either the depth buffer or the stencil buffer of a render target that does not have an attached depth buffer.

b) Try to clear the stencil buffer when the depth buffer does not contain stencil data.

Might be b) if not try inspecting the return value of the call, and set the dx debugoutput to max and see if it tells you anything.

Are you sure the scene you are using are using stencils currently ?

Rgs.
The noble art of losing face.May one day save the human race.And turn into eternal merit. What weaker minds might call disgrace.
oxygen728
oxygen728
Well... I'm not quire sure what you mean by A).. and I'm pretty sure I accomplish B).

here is my device initialization code... I think the problem has something to do with it.

    // Create the D3D object.    if( NULL == ( g_pD3D = Direct3DCreate9( D3D_SDK_VERSION ) ) )        return E_FAIL;    // Set up the structure used to create the D3DDevice. Since we are now    // using more complex geometry, we will create a device with a zbuffer.    D3DPRESENT_PARAMETERS d3dpp;     ZeroMemory( &d3dpp, sizeof(d3dpp) );    d3dpp.Windowed = TRUE;    d3dpp.SwapEffect = D3DSWAPEFFECT_DISCARD;    d3dpp.BackBufferFormat = D3DFMT_UNKNOWN;    d3dpp.EnableAutoDepthStencil = TRUE;    d3dpp.AutoDepthStencilFormat = D3DFMT_D16;	  d3dpp.PresentationInterval = D3DPRESENT_INTERVAL_IMMEDIATE; // Doesn't wait for Verticle Retrace!    // Create the D3DDevice    if( FAILED( g_pD3D->CreateDevice( D3DADAPTER_DEFAULT, D3DDEVTYPE_HAL, hWnd,                                      D3DCREATE_SOFTWARE_VERTEXPROCESSING,                                      &d3dpp, &g_pd3dDevice ) ) )    {        return E_FAIL;    }
Namethatnobodyelsetook
Namethatnobodyelsetook
D16 has no stencil bits. Use D24S8, or D15S1.
edit: Check to see which (if any) of these formats are supported at runtime with CheckDepthStencilMatch and CheckDeviceFormat
Dave Hunt
Dave Hunt
You've specified a depth/stencil format that only has a depth buffer. You need one of the depth formats that includes a stencil format, such as D3DFMT_D24S8 (24-bit depth plus 8-bit stencil).
Dave Hunt
Dave Hunt
Doh! Namethatnobodyelsetook strikes again!
Namethatnobodyelsetook
Namethatnobodyelsetook
Quote:
Original post by Dave Hunt
Doh! Namethatnobodyelsetook strikes again!

Only because the second sentence didn't occur to me until after I hit reply... Once I was done editing, you'd replied already. You've got to be quick around here. :)
oxygen728
oxygen728
Much appreciated! I'll see how this works!
oxygen728
oxygen728
Thanks guys! That helps!

Well.. Now it's a matter of actually having my shadow volume rendered and drawn properly =)

Atleast my scene isn't all goofed up =)

My frame rate is down a bunch... so that is a good sign that some shadows are trying to be drawn... they just aren't showing up.

If you like pain & are extremely, extremely bored for some reason.. I'll go ahead and post my shadow rendering/drawing function. If you look at it... don't make fun of me! This is a "rough draft" of what is to come =)

Again, I don't expect anybody to wade through this and tell me what's wrong... but if you are bored... =)

   // Disable z-buffer writes (note: z-testing still occurs), and enable the    // stencil-buffer    pd3dDevice->SetRenderState( D3DRS_ZWRITEENABLE,  FALSE );    pd3dDevice->SetRenderState( D3DRS_STENCILENABLE, TRUE );    // Dont bother with interpolating color    pd3dDevice->SetRenderState( D3DRS_SHADEMODE,     D3DSHADE_FLAT );    // Set up stencil compare fuction, reference value, and masks.    // Stencil test passes if ((ref & mask) cmpfn (stencil & mask)) is true.    // Note: since we set up the stencil-test to always pass, the STENCILFAIL    // renderstate is really not needed.    pd3dDevice->SetRenderState( D3DRS_STENCILFUNC,  D3DCMP_ALWAYS );    pd3dDevice->SetRenderState( D3DRS_STENCILZFAIL, D3DSTENCILOP_KEEP );    pd3dDevice->SetRenderState( D3DRS_STENCILFAIL,  D3DSTENCILOP_KEEP );    // If ztest passes, inc/decrement stencil buffer value    pd3dDevice->SetRenderState( D3DRS_STENCILREF,       0x1 );    pd3dDevice->SetRenderState( D3DRS_STENCILMASK,      0xffffffff );    pd3dDevice->SetRenderState( D3DRS_STENCILWRITEMASK, 0xffffffff );    pd3dDevice->SetRenderState( D3DRS_STENCILPASS,      D3DSTENCILOP_INCR );    // Make sure that no pixels get drawn to the frame buffer    pd3dDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, TRUE );    pd3dDevice->SetRenderState( D3DRS_SRCBLEND,  D3DBLEND_ZERO );    pd3dDevice->SetRenderState( D3DRS_DESTBLEND, D3DBLEND_ONE );        // With 2-sided stencil, we can avoid rendering twice:    pd3dDevice->SetRenderState( D3DRS_TWOSIDEDSTENCILMODE, TRUE );    pd3dDevice->SetRenderState( D3DRS_CCW_STENCILFUNC,  D3DCMP_ALWAYS );    pd3dDevice->SetRenderState( D3DRS_CCW_STENCILZFAIL, D3DSTENCILOP_KEEP );    pd3dDevice->SetRenderState( D3DRS_CCW_STENCILFAIL,  D3DSTENCILOP_KEEP );    pd3dDevice->SetRenderState( D3DRS_CCW_STENCILPASS, D3DSTENCILOP_DECR );    pd3dDevice->SetRenderState( D3DRS_CULLMODE,  D3DCULL_NONE );    // Draw both sides of shadow volume in stencil/z only    // Set stream source and all that good crap    pd3dDevice->SetFVF( D3DFVF_XYZ );    pd3dDevice->SetStreamSource(0, VB, 0, sizeof(VERTEX_PARTICLE));    D3DXMATRIXA16 mat, mat2;    D3DXMatrixScaling(&mat,50,500,50);    D3DXMatrixTranslation(&mat2,0,0,500);    D3DXMatrixMultiply(&mat, &mat, &mat2);    pd3dDevice->SetTransform( D3DTS_WORLD, &mat );    pd3dDevice->DrawPrimitive( D3DPT_TRIANGLELIST, 0, NumOfVertices / 3 );    pd3dDevice->SetRenderState( D3DRS_TWOSIDEDSTENCILMODE, FALSE );    // Restore render states    pd3dDevice->SetRenderState( D3DRS_SHADEMODE, D3DSHADE_GOURAUD );    pd3dDevice->SetRenderState( D3DRS_CULLMODE,  D3DCULL_CCW );    pd3dDevice->SetRenderState( D3DRS_ZWRITEENABLE,     TRUE );    pd3dDevice->SetRenderState( D3DRS_STENCILENABLE,    FALSE );    pd3dDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, FALSE );    // DRAW SHADOW        // Set renderstates (disable z-buffering, enable stencil, disable fog, and    // turn on alphablending)    pd3dDevice->SetRenderState( D3DRS_ZENABLE,          FALSE );    pd3dDevice->SetRenderState( D3DRS_STENCILENABLE,    TRUE );    pd3dDevice->SetRenderState( D3DRS_FOGENABLE,        FALSE );    pd3dDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, TRUE );    pd3dDevice->SetRenderState( D3DRS_SRCBLEND,  D3DBLEND_SRCALPHA );    pd3dDevice->SetRenderState( D3DRS_DESTBLEND, D3DBLEND_INVSRCALPHA );    pd3dDevice->SetTextureStageState( 0, D3DTSS_COLORARG1, D3DTA_TEXTURE );    pd3dDevice->SetTextureStageState( 0, D3DTSS_COLORARG2, D3DTA_DIFFUSE );    pd3dDevice->SetTextureStageState( 0, D3DTSS_COLOROP,   D3DTOP_MODULATE );    pd3dDevice->SetTextureStageState( 0, D3DTSS_ALPHAARG1, D3DTA_TEXTURE );    pd3dDevice->SetTextureStageState( 0, D3DTSS_ALPHAARG2, D3DTA_DIFFUSE );    pd3dDevice->SetTextureStageState( 0, D3DTSS_ALPHAOP,   D3DTOP_MODULATE );    // Only write where stencil val >= 1 (count indicates # of shadows that    // overlap that pixel)    pd3dDevice->SetRenderState( D3DRS_STENCILREF,  0x1 );    pd3dDevice->SetRenderState( D3DRS_STENCILFUNC, D3DCMP_LESSEQUAL );    pd3dDevice->SetRenderState( D3DRS_STENCILPASS, D3DSTENCILOP_KEEP );     //Draw a big, gray square    pd3dDevice->SetFVF( SHADOWVERTEX::FVF );    pd3dDevice->SetStreamSource( 0, SquareVB, 0, sizeof(SHADOWVERTEX) );    pd3dDevice->DrawPrimitive( D3DPT_TRIANGLESTRIP, 0, 2 );     //Restore render states    pd3dDevice->SetRenderState( D3DRS_ZENABLE,          TRUE );    pd3dDevice->SetRenderState( D3DRS_STENCILENABLE,    FALSE );    /////pd3dDevice->SetRenderState( D3DRS_FOGENABLE,        TRUE );    pd3dDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, FALSE );


My first guess is that something is wrong with the scaling / translation of my shadow volume.. But let's assume it is scaled properly and in the right position =)

Back to work for me... I'm determined to get these shadows going!

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.