Skip to main content
GameDev.net gamedev.net
🔒 Locked 🎮 Unity

DX9 Variance SM - Texture Hardware Filter Issues!

Started by jonathantompson Mar 15, 2011 at 8:00 PM 4 replies 3.5k views
Original Post
jonathantompson
jonathantompson
I am putting together a DX9 renderer with cascade variance shadow maps. Even though the shadows are rendering correctly, I am having trouble getting any sort of texture filtering to work. The aim of course (and the whole point of using the variance shadow maps) is to soften and anti-alias the projected shadow map with hardware sampling... I understand the theory that storing 2 moments and using Chebyshev inequality to approximate shadow percentage allows you to do linear sampling (which can be hardware accelerated).

The shadow map texture format is D3DFMT_G32R32F, though I have tried others... However no matter what sampler_state I include in the effect, I can't seem to get any visible hardware sampling.

I've included all the relevant HLSL code below as well as a few screen shots. I'm using a single low res shadow map to accentuate the aliasing.

Basically, I'm unsure why "Mag/MagFilter = xxx;" (POINT, LINEAR, ANISOTROPIC) doesn't affect the texture sampler.

Any help would be greatly appreciated:

Here is the vertex shader code TO BUILD THE SHADOW MAP:

Depth_PSIn BuildShadowMapVS(float3 Position : POSITION ) // Object space position
{
Depth_PSIn Output;

Output.Position = mul(float4(Position, 1), gWVP);
Output.PosView = mul(float4(Position, 1), gWV).xyz;

return Output;
}




Here is the pixel shader code TO BUILD THE SHADOW MAP:

float linstep(float min, float max, float v)
{
return clamp((v - min) / (max - min), 0, 1);
}

// Rescale into [0, 1]
float RescaleDistToLight(float Distance)
{
return linstep(gLight.nearFar.x, gLight.nearFar.y, Distance);
}

float2 GetFPBias()
{
//return float2(0.5, 0);
return float2(0, 0);
}

float2 ComputeMoments(float Depth)
{
float dx = ddx(Depth);
float dy = ddy(Depth);

// Compute first few moments of depth
float2 Moments;
Moments.x = Depth;
Moments.y = Depth * Depth + 0.25*(dx*dx + dy*dy); ;

return Moments;
}



float4 BuildShadowMapPS(Depth_PSIn Input) : COLOR
{
float Depth = RescaleDistToLight(length(Input.PosView)) + gVSMDepthEpsilon;
float2 Moments = ComputeMoments(Depth) + GetFPBias();
return float4(Moments.x, Moments.y, 0.0f, 0.0f);
}



Here is the vertex shader code WHEN RENDERING A TEXTURED MESH:

void MeshTextured_SpotLight_VS(float3 posL : POSITION,
float3 normalL : NORMAL0,
float2 tex0 : TEXCOORD0,
out float4 oPosH : POSITION0,
out float3 oPosW : TEXCOORD0,
out float3 oNormalW : TEXCOORD1,
out float3 oToEyeW : TEXCOORD2,
out float2 oTex0 : TEXCOORD3,
out float oSliceDepth : TEXCOORD4)
{
// Transform to homogeneous clip space.
oPosH = mul(float4(posL, 1.0f), gWVP);

// Transform vertex position to world space.
oPosW = mul(float4(posL, 1.0f), gW).xyz;

// Transform normal to world space (assume no non-uniform scaling).
oNormalW = mul(float4(normalL, 0.0f), gW).xyz;

// Compute the unit vector from the vertex to the eye.
oToEyeW = gEyePosW - oPosW;

// Pass on texture coords to PS
oTex0 = tex0;

// Calculate the slice depth
oSliceDepth = oPosH.z;
}



Here is the pixel shader code WHEN RENDERING A TEXTURED MESH:

// Per-pixel shading. Diffuse, ambient and specular
void PerPixelShading_SpotLight( float3 posW,
float3 normalW,
float3 toEyeW,
float3 lightVecW,
float4 color,
out float3 spec,
out float3 diffuse,
out float3 ambient,
out float spot)
{
// Compute the reflection vector.
float3 r = reflect(-lightVecW, normalW);

// Determine how much (if any) specular light makes it into the eye.
float t = pow(max(dot(r, toEyeW), 0.0f), gMtrl.specPower);

// Determine the diffuse light intensity that strikes the vertex.
float s = max(dot(lightVecW, normalW), 0.0f);

// Compute the ambient, diffuse and specular terms separately.
spec = t*(gMtrl.spec*gLight.spec).rgb;
diffuse = s*(gMtrl.diffuse.rgb*gLight.diffuse.rgb);
ambient = gMtrl.ambient.rgb*gLight.ambient.rgb;

// Compute spotlight coefficient.
spot = pow(max( dot(-lightVecW, gLight.dirW), 0.0000001f), gLight.spotPower);
}

// Per-pixel Shadow.
void PerPixelShadowing_SpotLight( float3 posW,
float sliceDepth,
float DistToLight,
out int Split,
out float shadowCoeff )
{
// Compute which split we're in:
// (slideDepth > dist_0) + (slideDepth > dist_1) + (slideDepth > dist_2) + (slideDepth > dist_3)
Split = dot(1, sliceDepth > gSplitDistances);

// Project using the associated matrix
float4 PosInLight = mul(float4(posW, 1), gSplitVPMatrices[Split]);
float2 LightTexCoord = (PosInLight.xy / PosInLight.w) * float2(0.5, -0.5) + 0.5;

// SHADOW CODE
if(gDoShadowing)
{
// Sample the correct shadow map
float2 Moments;
if(Split == 0)
Moments = tex2D(ShadowMapS0, LightTexCoord).xy;
if(Split == 1)
Moments = tex2D(ShadowMapS1, LightTexCoord).xy;
if(Split == 2)
Moments = tex2D(ShadowMapS2, LightTexCoord).xy;
if(Split == 3)
Moments = tex2D(ShadowMapS3, LightTexCoord).xy;
if(Split == 4)
Moments = tex2D(ShadowMapS4, LightTexCoord).xy;
Moments = Moments + GetFPBias();
float RescaledDist = RescaleDistToLight(DistToLight);

// VARIANCE SHADOW MAPS
shadowCoeff = ChebyshevUpperBound(Moments, RescaledDist, gVSMMinVariance);
shadowCoeff = LBR(shadowCoeff);
}
else
{
shadowCoeff = 1.0f;
}
}

float4 MeshTextured_SpotLight_PS(float3 posW : TEXCOORD0,
float3 normalW : TEXCOORD1,
float3 toEyeW : TEXCOORD2,
float2 tex0 : TEXCOORD3,
float sliceDepth : TEXCOORD4) : COLOR
{
// Interpolated normals can become unnormal--so normalize.
normalW = normalize(normalW);
toEyeW = normalize(toEyeW);

// Calculate normalized light vector and distance to light
float3 lightVecW = gLight.posW - posW;
float DistToLight = length(lightVecW);
lightVecW /= DistToLight;

// Sample Texture map.
float4 texColor = tex2D(TexS, tex0);

// Calculate per-pixel shading for spot light
float3 spec;
float3 diffuse;
float3 ambient;
float spot;
PerPixelShading_SpotLight(posW, normalW, toEyeW, lightVecW, texColor, spec, diffuse, ambient, spot);

// Calculate Shadow
int Split;
float shadowCoeff;
PerPixelShadowing_SpotLight(posW, sliceDepth, DistToLight, Split, shadowCoeff);

// Light/Texture pixel. Note that shadow coefficient only affects diffuse/spec.
float3 litColor = spot*(ambient*texColor.rgb + shadowCoeff*(diffuse*texColor.rgb + spec));

// Visualize the splits by adding a linearly interpolated color offset
if (gVisualizeSplits) {
litColor = lerp(litColor, SplitColors[Split], 0.5);
}

return float4(litColor, gMtrl.diffuse.a*texColor.a);
}


POINT Filter, 1 cascade, 512x512 textures, No texture blur:
5529568603_e6566cf386.jpg



LINEAR Filter, 1 cascade, 512x512 textures, No texture blur:
5529568217_f22450507c.jpg



ANISOTROPIC Filter, 1 cascade, 512x512 textures, No texture blur:
5530155428_5bf7bc1008.jpg

POINT Filter, 4 cascades, 512x512 textures, 5x5 box blur:
5530156448_c6e9df444b.jpg
glaeken
glaeken
Haven't looked at the code much but are you sure you video card supports floating point filtering for RG32F? Check the caps to make sure.
jonathantompson
jonathantompson

Haven't looked at the code much but are you sure you video card supports floating point filtering for RG32F? Check the caps to make sure.


You might be on the right track. Though my work computer doesn't list D3DUSAGE_QUERY_FILTER, my home computer does. My rendering results I get are the same on the computer that does. Maybe some other issue?

My work computer caps doesn't seem to list hardware filtering for RG32F... (windows XP):
5529813157_2ee5cee845_b.jpg


However, on my home computer it does list support for G32R32F filtering (windows 7):
5529822105_144bb9fe8d_b.jpg
jonathantompson
jonathantompson

Haven't looked at the code much but are you sure you video card supports floating point filtering for RG32F? Check the caps to make sure.


I guess a potential follow up question would be: If RG32F is not available, then what format do people use for variance shadow map textures?

When I try 16bit floating point formats I get HORRIBLE artifacts... Is the only option then to manually filter bi or trilinearly in the fragment shader? If this is the case, then I don't see the advantage over regular shadow maps and PCF...
glaeken
glaeken

[quote name='glaeken' timestamp='1300223011' post='4786190']
Haven't looked at the code much but are you sure you video card supports floating point filtering for RG32F? Check the caps to make sure.


I guess a potential follow up question would be: If RG32F is not available, then what format do people use for variance shadow map textures?

When I try 16bit floating point formats I get HORRIBLE artifacts... Is the only option then to manually filter bi or trilinearly in the fragment shader? If this is the case, then I don't see the advantage over regular shadow maps and PCF...
[/quote]

One of the benefits is that you can run a separable filter over the shadowmap, like a separable gaussian blur.
jonathantompson
jonathantompson
Just as an update to anyone who come across this:

I was never able to work out how to get floating point texture filtering on my 7800GTX machine working. Caps says it is supported. When enabled no DX9 warning or error is thrown, yet it just doesn't work.

I've tried the same code on other cards (an ATI 5850) and the filtering looks great. For now, I'm using bilinear filtering in the shader on the 7800GTX machine as a work around:


// Bilinear interpolation texture lookup
float4 tex2DBilinear( sampler textureSampler, float2 uv )
{
float4 tl = tex2D(textureSampler, uv);
float4 tr = tex2D(textureSampler, uv + float2(gTexelSize, 0));
float4 bl = tex2D(textureSampler, uv + float2(0, gTexelSize));
float4 br = tex2D(textureSampler, uv + float2(gTexelSize , gTexelSize));
float2 f = frac( uv.xy * gTextureSize ); // get the decimal part
float4 tA = lerp( tl, tr, f.x ); // will interpolate the red dot in the image
float4 tB = lerp( bl, br, f.x ); // will interpolate the blue dot in the image
return lerp( tA, tB, f.y ); // will interpolate the green dot in the image
}



It's slower, but at least I get some shadow map filtering!

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.