Skip to main content
gamedev.net
🔒 Locked

Shadow Map 3x3 problem

Started by Luna_White Feb 22, 2009 at 2:11 PM 12 replies 1.6k views
Original Post
Luna_White
Luna_White
Hello! I am trying to create a shadow map shader, from the code in the book Introduction to DirectX 9 a Shader Approach and the soft shadow tutorial in here. It worked fine with the 2x2 PCF but I am trying to make it 3x3, following the Soft Shadow tutorial on GameDev. I have this: // 3x3 percentage closest filter. float dx = 1.0f / SMAP_SIZE; float4 vTexCoords[9]; // Sample each of them checking whether the pixel under test is shadowed or not vTexCoords[0] = projTex; vTexCoords[1] = projTex + float2( -dx, 0.0f ); vTexCoords[2] = projTex + float2( dx, 0.0f ); vTexCoords[3] = projTex + float2( 0.0f, dx ); vTexCoords[4] = projTex + float2( 0.0f, -dx ); vTexCoords[5] = projTex + float2( dx, dx ); vTexCoords[6] = projTex + float2( -dx, dx ); vTexCoords[7] = projTex + float2( dx, -dx ); vTexCoords[8] = projTex + float2( -dx, -dx); float fShadowTerms[9]; float fShadowTerm = 0.0f; for( int i = 0; i < 5; i++ ) { float A = tex2D(ShadowMapS , vTexCoords ).r; // Texel is shadowed fShadowTerms = (A + SHADOW_EPSILON < depth) ? 0.0f : 1.0f; fShadowTerm += fShadowTerms; } // Get the average fShadowTerm /= 9.0f; float3 litColor = spot*ambient*texColor.rgb + spot* fShadowTerm *(diffuse*texColor.rgb + spec); return float4(litColor, Material.diffuse.a*texColor.a); Implemented in the shader, but it fails the screen is black if the loop i is bigger than 5. Hopefully someone can find what is wrong with this ^^ Thanks! :) [Edited by - Luna_White on February 22, 2009 2:39:16 PM]
Luna_White
Luna_White
I just noticed I left the loop to be lesser than 5 when copy pasted but fixed the average calculation.
The i shown here is a left over, the get average I calculated with 5 for this, 9 only if the loop is set to 9.
Luna_White
Luna_White
Weird! changing the pixel and vertex shader to version 3 made it work! (not black screen anymore!)

but.. the colors are a bit weird, as if brighter and it is making the shadows look weirdly colored also.
Luna_White
Luna_White
Also, with shader 3, the edges of things appear lit when they should not be..
MJP
MJP
Debug, debug, debug! Run your app with the debug runtimes active, and see if you get any warnings or errors. Use PIX to step through your shader and see why you're getting unexpected results.
jpventoso
jpventoso
Did your shader compile under sm20? Maybe you're exceding the max instruction slots for sm20.
The shader itself seems to be good.
Luna_White
Luna_White
What do you mean max instruction slots?

It fails to compile if I put the loop higher than 5.
It works quite fine on 3 (I forgot to revert things back from all the messing around)
jpventoso
jpventoso
My knowledge is more practical than theorical on this matter, so the terms I use could not be correct :) (Plus, my english is bad) But I'll try to explain it.

Every operation done on your shader code uses instruction slots. There's a limit of instructions slots that varies with the shader version you're compiling.

From the SDK docs:

ps_1_1, ps_1_2, ps_1_3: 4 texture + 8 arithmetic
ps_1_4: 6 texture + 8 arithmetic per phase
ps_2_0: 32 texture + 64 arithmetic
ps_2_x: 96 minimum, and up to the number of slots in D3DCAPS9.D3DPSHADERCAPS2_0.NumInstructionSlots
ps_3_0: 512 minimum, and up to the number of slots in D3DCAPS9.MaxPixelShader30InstructionSlots

So, if your code uses more than 32 texture instructions or 64 arithmetic instructions, it won't be compiled on that target. The solution is to reduce the amount of instructions, or compile the shader on a higher target (ps_3_0 in your case).

Hope this helps!
GMano
GMano
As others said, you are probably running into the instruction count limit for ps2. You can do a 3x3 filter using only 4 samples (with linear filtering on) on texel boundaries and weighting them, which might put you back below the instruction limit. i.e.

vTexCoords[0] = projTex + float2( -dx/2, -dx/2 );//this gets average of ul,uc,lc and c texels - weight by 4/9

vTexCoords[1] = projTex + float2( dx, -dx/2 );//this gets average of ur and rc texel - weight by 2/9

vTexCoords[2] = projTex + float2( -dx/2, dx );//this gets average of bl and bc texels - weight by 2/9

vTexCoords[3] = projTex + float2( dx, dx );//bottom right texel - weight by 1/9


Also, if you are using nVidia hardware, you can get pcf filtering basically for free by using hardware shadow mapping, or if you are using ati hardware you can use the fetch4 command to sample 4 points at a time.
Luna_White
Luna_White
Thank you GMano!
Just a question, what do you mean by weighting them? sorry I am not a native english speaker and it might be it :)
GMano
GMano
Quote:
Original post by Luna_White
Thank you GMano!
Just a question, what do you mean by weighting them? sorry I am not a native english speaker and it might be it :)



Multiply the results that you get back from your sampler by the weights. In your original code, every sample represented 1 texel, you took 9 samples, so you multiply each sample by 1/9 and add together (you divided by 9 at the end, which is the same thing). In my example, the samples represent 1, 2 or 4 texels, so you multiply by 1/9, 2/9, or 4/9 respectively (or you could multiply by 1, 2 or 4 and divide total sum by 9).
Luna_White
Luna_White
Thank you very much again :)
jpventoso
jpventoso
Quote:
Original post by GMano
Also, if you are using nVidia hardware, you can get pcf filtering basically for free by using hardware shadow mapping, or if you are using ati hardware you can use the fetch4 command to sample 4 points at a time.


That's a good way to gain both performance and quality. I implemented hardware shadow mapping with great results. It works on most newer ATI cards too.

If you're interested in this optimization, I can give you some hints.

Good luck!

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.