🔒 Locked

hlsl normal bug - vs - ps

Started by bkenwright Dec 8, 2005 at 5:36 PM 5 replies 1.1k views

Original Post

Dec 8, 2005 at 5:36 PM

Hey Everyone, well I'm not sure about this...maybe I'm just not seeing it...but it just seems like a wierd problem this. Just passing a normal from the vertex shader to the pixel shader. If I set the normal to a constant in vertex shader fx file like this: OUT.normal = float4(0 , 0, -1.0f , 0); And in my pixel shader function, I do this: float4 FinalColour = float4(1,0,0,1); float4 light = float4( 0.0f, 0.0f, -1.0f, 0.0f ); FinalColour = FinalColour * dot(IN.normal, light); It should produce a FinalColour output of red!...as the dot product of the light and normal should produce 1! But it doesn't. Its black! This is the problem, if I set the normal to (0,0,1,0) in the vertex function and set the light direction to (0,0,1,0) in the pixel shader the dot product seems to work okay. Also if I do a small variable in the pixel shader function like so: float4 norm = float4( 0.0f, 0.0f, -1.0f, 0.0f ); And do the dot product of the two, dot(light,norm) where there both negative, this works as well....just when the value in the vertex shader is negative. I'm only doing a z value, as I'm testing for a simple light direction, which is into the screen. Hope I've explained it okay...just not right? I mean maybe I'm missing something...maybe theres a maths thing...or a thing with shaders on passing negative values? Any ideas on this? Or how to maybe fix it? What to try next? Thanks, Ben Little details on the structurs I use to pass the data // This isn't really needed as we are just setting it to constant for debugging struct vertexInput { float3 position : POSITION; float3 normal : NORMAL; float2 texCoordDiffuse : TEXCOORD0; float2 texCoord2 : TEXCOORD1; }; // Set to float4 for a normal!...should be okay. struct vertexOutput { float4 HPOS : POSITION; float4 normal : TEXCOORD0; float4 T0 : TEXCOORD1; // fur alpha float4 T1 : TEXCOORD2; // fur offset / interfur shadowing };

jollyjeffers

Dec 8, 2005 at 5:51 PM

The mathematical definition of (0,1,0)•(0,-1,0) should be -1 (typically clamped to 0). If you draw it as quality ascii art:

  •  |  |  V  ^  |  |-----

they point at each other. a DP of 1 should have them pointing in the same direction.

Try inverting one of them and you should be sorted [smile]

hth
Jack

bkenwright

Dec 8, 2005 at 6:01 PM

yup, they work if there (0,0,1)•(0,0,1), but won't work if I pass -1, and do(0,0,-1)•(0,0,-1) shouldn't this be 1?

Or are negative values passed from the vertex shader in the normal just clamped to 0->1?

If I mix (0,0,-1)•(0,0,1) and (0,0,1)•(0,0,-1)...that gives me all black rendering...but I want a positive value.

Just a strange result in the shader and wanted to know why its doing it.

S1CA

Dec 8, 2005 at 6:38 PM

My guess is you're using ps1.1 as your pixel shader target...

...in ps1.1, your "normal" will be passed in a texture coordinate (it's either there or a colour).

Under ps1.1, a texture coordinate used directly rather than sampling a texture (i.e. using the "texcoord" instruction) has its elements clamped to the range 0 to 1.

My advice is this, for 1.x targets, it's a good idea to learn ps1.x pixel shader assembly, for three reasons:

1) so that you know which HLSL will and won't be possible (9 instruction slots isn't many...)

2) so you can see what the HLSL compiler has produced from your input; this is also useful for "massaging" the compiler into doing what you want.

3) so you can rewrite your shader in asm if you can't persuade the compiler to do what you want (personally, for 1.x pixel shader targets, I'd use asm - HLSL can be a good start to prototype the effect, then hand optimise out any silly things).

Try this:

float3 normal = IN.normal;
normal = (normal - 0.5) * 2.0f;
...
// use normal as you like here

That's an example of what I mean by massaging; it's to persuade the compiler to use the _bx2 modifier in the asm. Spliiting things onto multiple lines and correct parenthesis can help guide it to a desirable answer [smile].

bkenwright

Dec 8, 2005 at 6:43 PM

I can only assume, that any values you pass in TEXCOORD0, TEXCOORD1..etc to the pixel shader stage from the vertex stage, have to be in the range 0 to 1?

Is this true?

S1CA

Dec 8, 2005 at 6:53 PM

Quote:
Original post by bkenwright
I can only assume, that any values you pass in TEXCOORD0, TEXCOORD1..etc to the pixel shader stage from the vertex stage, have to be in the range 0 to 1?

Is this true?

The only interpolated values you have available in a 1.x pixel shader are colours (diffuse and specular) and texture coordinates. Things passed in texture coordinates are clamped to a 0 to 1 range by the "texcoord" instruction - in the asm, that must occur before the dot product and any biasing, so in practical terms, you can assume that the inputs have already been clamped before they came in to your HLSL pixel shader function.

That's why ps1.1 has the _bx2, bias, invert and negate modifiers at the asm level: so you can can treat unsigned inputs (i.e. stuff clamped to the 0 to 1 range) as signed values.

The HLSL compiler inserts the dot product and assumes you're already aware that the inputs are clamped and need biasing if you want them treated as signed.

out = 2*(in-0.5) or similar is the way to go.

An alternative solution for you depending on your target hardware might be to go to ps2.0 (or ps1.3/ps1.4 if you can exclude GeForce3 from your target hardware/market).

bkenwright

Dec 8, 2005 at 7:00 PM

Hi S1CA,
shows you learn something new every day. Thanks for your help on that, its really clarified it, shows I should pay extra attention to the details when reading the shader docs. Got so used to hlsl I forget about how tricky and sneeky the ps1.1 asm was.

Thanks agian,

Ben

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.