HLSL Weird behavior

How are you reading these interpolated values back out? The vertex shader runs on the GPU (assuming hardware vertex processing) and can't (as far as I know) get data back to the CPU.

Anything you store in a TEXCOORD register to be passed to the pixel shader will be interpolated between vertices. If you're getting those values by simply spitting out the .a value to the render target, that would explain it.

Evil Steve:

The shader writes to a texture set as a render target and then i copy the texture into the system memory using Device.GetRenderTargetData().

MJP:

your guess is right, I am outputing to a TEXCOORD.

let me show you the code in full:

struct a2v
{
float4 Position : POSITION;
};

struct v2p
{
float4 Position : POSITION;
float4 worldPosition : TEXCOORD0;
};

struct p2s
{
float4 Color : COLOR;
};


void vsLin( in a2v IN, out v2p OUT )
{
OUT.Position = mul(IN.Position, worldViewProjection);
float4 worldPos = mul(IN.Position, worldMatrix);

if( ( worldPos.z > 0 ) && ( abs( worldPos.z - cubeVertex ) < 0.01 ) )
{
OUT.worldPosition.a = 0;
OUT.worldPosition.r = 1;
OUT.worldPosition.g = (worldPos.x + cubeVertex) / cubeDimensions;
OUT.worldPosition.b = (cubeVertex - worldPos.y) / cubeDimensions;
}
else if( ( worldPos.z < 0 ) && ( abs( worldPos.z + cubeVertex ) < 0.01 ) )
{
OUT.worldPosition.a = 0;
OUT.worldPosition.r = 2;
OUT.worldPosition.g = (cubeVertex - worldPos.x) / cubeDimensions;
OUT.worldPosition.b = (cubeVertex - worldPos.y) / cubeDimensions;
}
else if( ( worldPos.x > 0 ) && ( abs( worldPos.x - cubeVertex ) < 0.01 ) )
{
OUT.worldPosition.a = 0;
OUT.worldPosition.r = 3;
OUT.worldPosition.g = (cubeVertex - worldPos.z) / cubeDimensions;
OUT.worldPosition.b = (cubeVertex - worldPos.y) / cubeDimensions;
}
else if( ( worldPos.x < 0 ) && ( abs( worldPos.x + cubeVertex ) < 0.01 ) )
{
OUT.worldPosition.a = 0;
OUT.worldPosition.r = 4;
OUT.worldPosition.g = (worldPos.z - cubeVertex) / cubeDimensions;
OUT.worldPosition.b = (cubeVertex - worldPos.y) / cubeDimensions;
}
else if( ( worldPos.y > 0 ) && ( abs( worldPos.y - cubeVertex ) < 0.01 ) )
{
OUT.worldPosition.a = 0;
OUT.worldPosition.r = 5;
OUT.worldPosition.g = (worldPos.x - cubeVertex) / cubeDimensions;
OUT.worldPosition.b = (worldPos.z + cubeVertex) / cubeDimensions;
}
else if( ( worldPos.y < 0 ) && ( abs( worldPos.y + cubeVertex ) < 0.01 ) )
{
OUT.worldPosition.a = 0;
OUT.worldPosition.r = 6;
OUT.worldPosition.g = (worldPos.x + cubeVertex) / cubeDimensions;
OUT.worldPosition.b = (cubeVertex - worldPos.z) / cubeDimensions;
}
else
{
OUT.worldPosition.a = 1;
OUT.worldPosition.r = 7;
OUT.worldPosition.g = 0;
OUT.worldPosition.b = 0;
}
}

void ps( in v2p IN, out p2s OUT)
{
OUT.Color = IN.worldPosition;
}


I just said before that I'm using the "a" and some simple number to simplify the problem but I guess i should've shown you guys the whole problem from the beginning.

What makes me wonder is that I had an older shader which is same to this but only outputs the world coordinates (worldPos is the code above). there wasn't any interpolation there!

I used TexCoord since as I remember using another POSITION element in the v2p structure would not make the shader compile...

do you know what should be done?

Thanks in advance...
Raed

BTW i changed worldPosition semantic from TEXCOORD to POSITION1 and the same problem is still there. I still get interpolated values.

something tells me its the HLSL's if statement implementation :(

Your if statement is fine, it's just the interpolation causing your results. You can confirm this if you want by debugging your shader in PIX, just find that draw call and look at the output for each vertex.

I *think* you may be able to use the COLOR register with flat-shading mode (set with IDirect3DDevice9::SetRenderState(D3DRS_SHADEMODE, D3DSHADE_FLAT)) if you want a non-interpolated register, but I'm not 100% on this.

This is not an error in the HLSL. If you look at the ps input registers for pre dx10 you will find that they are all interpolated between vertices when rendering polygons. The same goes for position registers. I'm assuming that you're rendering polygons because points would not have interpolation issues.

Your only option for maintaining a single value uninterpolated is to turn on flat shading and then use the color registers. Your polygons will then have the color value specified by the first vertex in the polygon winding order. When you turn on flat shading both (ps_3_0) color registers will not be interpolated. Of course this means that the values from the other two vertices in your triangle will be ignored.

Use the following statement to sent the render state to flat shader mode.

IDirect3DDevice9::SetRenderState( D3DRS_FILLMODE, D3DSHADE_FLAT );

So MJP is correct.

I have no idea if this is applicable or not, but what about the "nointerpolation" keyword? According to the docs:

Quote:

MSDN nointerpolation - Do not interpolate the outputs of a vertex shader before passing them to a pixel shader.

The nointerpolate modifier was add in HLSL 4.0. You'll need a dx10 system to get that to work.

Hi guys

i set the shading mode to flat and it did not work. What makes me really doubt that the problem is some where else is that the following shader perfectly outputs the intended positions:

void vsLin( in a2v IN, out v2p OUT )
{
OUT.Position = mul(IN.Position, worldViewProjection);
float4 worldPos = mul(IN.Position, worldMatrix);

OUT.worldPosition.a = 0;
OUT.worldPosition.r = worldPos.x;
OUT.worldPosition.g = worldPos.y;
OUT.worldPosition.b = worldPos.z;
}

void ps( in v2p IN, out p2s OUT)
{
OUT.Color = IN.worldPosition;
}

I just can't understand why this one works and the other one doesn't!

To give you guys a better picture of what is going on, i am rendering a cube as a second pass to a texture, here is the code:

private void RenderToWorldCoorTexture()
{
using (Surface surface = worldCoordTexture.GetSurfaceLevel(0))
{
rts.BeginScene(surface);
Device.Clear(ClearFlags.ZBuffer | ClearFlags.Target, 0x00000000, 1.0f, 0);
Device.SetStreamSource(0, vertices, 0);
Device.VertexFormat = CustomVertex.PositionTextured.Format;

worldCoordEffect.SetValue("cubeVertex", cubeVertex);
worldCoordEffect.SetValue("cubeDimensions", cubeDimensions);

int numPasses = worldCoordEffect.Begin(FX.None);

foreach (int faceValue in Enum.GetValues(typeof(CubicPanoramaFace)))
{
CubicPanoramaFace face = (CubicPanoramaFace)faceValue;

switch (face)
{
case CubicPanoramaFace.Front:
WorldMatrix = Matrix.Translation(0, 0, 0);
break;

case CubicPanoramaFace.Back:
WorldMatrix =
Matrix.RotationY(Geometry.DegreeToRadian(180)) * Matrix.Translation(0, 0, 0);
break;
.... // and so on with the rest of the cube faces
}

for (int pass = 0; pass < numPasses; pass++)
{
worldCoordEffect.BeginPass(pass);

Device.SetTexture(0, cubeTextures[face]);
worldCoordEffect.SetValue("worldMatrix", WorldMatrix);
worldCoordEffect.SetValue("worldViewProjection",
WorldMatrix * ViewMatrix * ProjectionMatrix);

Device.DrawPrimitives(PrimitiveType.TriangleList, 0, 2);

worldCoordEffect.EndPass();
}
}

worldCoordEffect.End();
rts.EndScene(Filter.Linear);
}
}


Thanks for your help