🔒 Locked

Cube maps with shaders

Started by acid2 Mar 16, 2005 at 11:53 AM 15 replies 2k views

Original Post

Mar 16, 2005 at 11:53 AM

Hi, It seems all I ever do is have problems... so sorry that I keep asking so many questions - you guys are brilliant at your responses though, I'm learning so much! Anyway, I'm giving shaders a real go atm, and I'm eager to try lots of things. I thought it would be a good idea to try and get cube mapping in. I have sort of got it working. When the camera is at 0, 0, 0 and the mesh is in front of the camera (0, 0, 50 in this case) then it seems to render ok:

However, if I move the camera back 10 units (so it is at 0, 0, -10) then I get this really bizzare result:

Any ideas why this is? I'm pretty unsure with my shader but I thought I had it down correctly. People have said I should use world-invert-transposed matrix, but it gives realllly bad results!


float4x4 matWIT : WorldInverseTranspose;
float4x4 matWVP : WorldViewProjection;
float4x4 matVI : ViewInverse;
float4x4 matW : World;

texture CubeTexture;
samplerCUBE cubeSampler = sampler_state
{
	texture = <CubeTexture>;
};

struct vsData
{
	float4 pos		: POSITION;
	float3 norm 	: TEXCOORD0;
	float3 viewDir	: TEXCOORD1;
};

vsData vs_CubeMap(	float4 pos : POSITION,
					float4 norm : NORMAL	)
{
	vsData Out;
	
	Out.pos = mul(pos, matWVP);
	Out.norm = mul(norm, matWIT);
	Out.viewDir = matVI[3].xyz - pos.xyz;
	
	return Out;
}

float4 fs_CubeMap(	vsData In	) : COLOR0
{
	float3 normalNormalized = normalize(In.norm);
	float3 viewDirNormalized = normalize(In.viewDir);

	return texCUBE(cubeSampler, reflect(viewDirNormalized, normalNormalized));
}

technique Cubemap
{
	pass p0
	{
		VertexShader = compile vs_1_1 vs_CubeMap();
		PixelShader = compile ps_2_0 fs_CubeMap();
	}
}

Edit: Oh, I thought it might help if you know the colours in the skybox there. Pink is forward, light blue is right, lime yellow is left, yellow is top, dark blue is bottom and red is backwards. [Edited by - acid2 on March 16, 2005 1:41:33 PM]

browny

Mar 16, 2005 at 12:31 PM

i am not sure what's wrong , but i could pick one lil mistake in ur code though.. i dunno whether that's the culprit of not.

When u transform a vertex normal.. u never multiply it with the modelview matrix...cuz modelview matrix has a translation component in it .. remember ? And u dont translate normal, u only rotate or orient them. Well therez a way to do that even with modelview matrix but letz not go into that... pplz usually transform normals using the InverseTranspose matrix

acid2

Mar 16, 2005 at 12:34 PM

I tryed it with the matWorldIT matrix (World Inverse Transpose) and I got a really nasty result (check this link)

blue_knight

Mar 16, 2005 at 12:45 PM

Make sure texture clamping is on for all 3 axises (U,V,W). Then transform your normal by the 3x3 rotational part of your world matrix (or the inverse transpose), as suggested. The reason your result is nasty seems to be the clamping mode (I bet you have it on WRAP which is the default).

superpig

Mar 16, 2005 at 12:56 PM

You can cause the normal to 'ignore' the translation component of the world matrix if you set it's w component to zero, i.e.

Out.norm = normalize(mul(float4(norm.xyz, 0.0), matW).xyz);

acid2

Mar 16, 2005 at 12:57 PM

I just set the addressu and v modes to clamp and it doesn't make any difference. So, that doesn't seem to be the problem :( Setting what Superpig suggested and using the IT matrix doesn't seem to make a difference. Here is an updated screenshot with what I'm currently getting:

Linky

It's like a perspective problem - reaaallly weird :\ Here's how I'm setting my matrices, if its any help:

Microsoft.DirectX.Matrix WorldIT = meshCube.SceneGraphNode.World;WorldIT.Transpose(Microsoft.DirectX.Matrix.Invert(meshCube.SceneGraphNode.World));cube.SetGlobal("matWIT", WorldIT);cube.SetGlobal("matWVP", meshCube.SceneGraphNode.World * camMain.View * camMain.Projection);cube.SetGlobal("matW", meshCube.SceneGraphNode.World);Microsoft.DirectX.Matrix ViewI = camMain.View; ViewI.Invert();cube.SetGlobal("matVI", ViewI);

blue_knight

Mar 16, 2005 at 1:17 PM

Did you clamp W?
Meaning texture clamping ADDRESS_W I think.

acid2

Mar 16, 2005 at 1:23 PM

Yea, I tryed clamping u v and w, and them individually

blue_knight

Mar 16, 2005 at 1:31 PM

OK, I just realized that you appear to be doing the reflection in your pixel shader. In that case you do NOT want to normalize you normal and view direction the vertex shader, but you DO want to normalize those 2 vectors in your pixel shader before you do the reflection calculation. It appears that you never normalize your view direction which would cause your reflection vector direction to be skewed.

acid2

Mar 16, 2005 at 1:40 PM

Normalizing the vectors had no effect.. I wonder what could be wrong - seemed to have tried nearly everything now. Thank you all for your patience in this, I'm determined to get it right >:( Oh, and I updated the sourcecode at the top with my current code.

blue_knight

Mar 16, 2005 at 1:50 PM

I'm thinking now that maybe that this line:
"Out.viewDir = matVI[3].xyz - pos.xyz;" maybe wrong. It appears that you are accessing your matVI matrix row major but maybe its stored column major (I could be wrong about this). Why not just pass in the world space camera position instead (Out.viewDir = worldSpaceCamPos - pos;)? That's the only other thing I can think of. Since you're doing the reflection/normalization in the pixel shader, your cubemap lookup should be accurate even on low-tesselated geometry.

acid2

Mar 16, 2005 at 1:53 PM

Nope, still being weird. That line was correct because I am using row-major matrices, and I've tried it witg my water fresnel effect. I did give your suggestion a try though, btw.

This is so frustrating :(

Tryed it on a normal cube, 2 triangles per face and this is the result... ouch.

blue_knight

Mar 16, 2005 at 2:05 PM

Well, did you try looking at your mesh normals? There really is not reason I can think of why your code shouldn't work. The only real flaw was not normalizing your view direction at some point (which you fixed). At this point I would start looking at your mesh data.

Edit-
Wait a minute, this line is subtraction LOCAL SPACE position, isn't it: matVI[3].xyz - pos.xyz;
That would be bad, it should subtract WORLD space position.

Kalidor

Mar 16, 2005 at 2:08 PM

Quote:
Out.norm = mul(norm, matWIT);
Out.viewDir = matVI[3].xyz - pos.xyz;

It looks to me like you are using the eye-space normal and world-space view vector. I'm not sure if that's the problem though, but maybe try something like this...

Out.viewDir = mul(matVI[3], matWVP).xyz - Out.pos.xyz;

EDIT: I gotta type faster. That's also assuming matVI[3] is in world-space, which I'm not sure about after reading blue_knight's edit above.

EDIT2: Added .xyz swizzle so that we're subtracting from a float3

acid2

Mar 16, 2005 at 2:13 PM

Already checked it, looks fine.. :( It's something to do with my engine, because if I run the shader in FX Composer it's 100% A-OK :) So that's me happy because I've written a shader, and me unhappy because my engine is broken :P

blue_knight

Mar 16, 2005 at 2:20 PM

OK, that makes sense. However, I do think that the this line:
Out.viewDir = matVI[3].xyz - pos.xyz;
should be
Out.viewDir = matVI[3].xyz - WorldPos.xyz;
(where WorldPos = pos multiplied by the world matrix). Right now it works since
the object is centered around (0,0,0) but as soon as you try to move it it'll break again.

acid2

Mar 16, 2005 at 2:21 PM

Yea, I actually added that myself, but forgot to update the code. Sorry to lead you all astray on that line.

Thanks so much for putting up with me, really helped me solve some problems that I would have had anyway! :) ++ rating's all round ;)

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.