🔒 Locked

Vertex Shader problem

Started by Asterisk Man Dec 17, 2004 at 10:50 AM 3 replies 650+ views

Original Post

Dec 17, 2004 at 10:50 AM

I got a slight problem: I set up my very simple vertex shader to use the diffuse color of a vertex. The problem comes when I set the vertex color to white (0xFFFFFFFF), which then shows up as a pinkish color, any thoughts as to why this happens? Here is the vertex definition: struct VERTEX { float x,y,z; DWORD color; float tu, tv; }; #define D3DFVF_VERTEX (D3DFVF_XYZ | D3DFVF_DIFFUSE | D3DFVF_TEX1) And the vertex shader I use: vs.1.1 dcl_position v0 dcl_color v1 m4x4 oPos, v0, c4 mov oD0, v1 Where c4 is the transformation matrix. Any thoughts would be appreciatied. EDIT: never mind, after tinkering with it for a few hours, I realized that if you set a texture to the device, it will alter the color of the object. No clue as to why though, but when I removed the texture from my objects, they became snow white. [Edited by - Asterisk Man on December 17, 2004 1:12:59 PM]

DrGUI

Dec 17, 2004 at 4:33 PM

Quote:
Original post by Asterisk Man
Any thoughts would be appreciatied.

EDIT: never mind, after tinkering with it for a few hours, I realized that if you set a texture to the device, it will alter the color of the object. No clue as to why though, but when I removed the texture from my objects, they became snow white.

You have texture co-ordinates defined in your shader - might that texture have pink in?

Asterisk Man

Author

Asterisk Man

Dec 17, 2004 at 10:27 PM

As a matter of fact it did. However, since I wasn't specifying any Texture coordinates, I wasn't expecting that the texture be applied (I'm still new at this stuff).

Muhammad Haggag

156

Muhammad Haggag

Dec 17, 2004 at 10:42 PM

Quote:
Original post by Asterisk Man
EDIT: never mind, after tinkering with it for a few hours, I realized that if you set a texture to the device, it will alter the color of the object. No clue as to why though, but when I removed the texture from my objects, they became snow white.

Can you show you texture stage states? This is a bug with your code. When you don't want D3D to use a texture, you have to tell it so explicitly:

device->SetTextureStageState(0, D3DTSS_COLOROP, D3DTOP_SELECTARG1);device->SetTextureStageState(0, D3DTSS_COLORARG1, D3DTA_DIFFUSE);device->SetTextureStageState(0, D3DTSS_ALPHAOP, D3DTOP_SELECTARG1);device->SetTextureStageState(0, D3DTSS_ALPHAARG1, D3DTA_DIFFUSE);device->SetTextureStageState(1, D3DTSS_COLOROP, D3DTOP_DISABLE);device->SetTextureStageState(1, D3DTSS_ALPHAOP, D3DTOP_DISABLE);

What this does is tell D3D to take color from the diffuse component in the vertex (i.e. totally ignore the texture).

Asterisk Man

Author

Asterisk Man

Dec 18, 2004 at 12:00 PM

Up until now I wasn't setting any texture stage states, I was currently implementing the material system (and learning about materials at the same time), when I decided to try and see what else I could do with vertex shaders. The way I have it implemented at the moment is: if it has a texture, render it, if it doesn't, dont.

However, I did run into another problem with the vs when trying to do directional lighting with it: If I transform the object while the light is hitting it, the the same vertices will stay lit. I realized that this was due to the vertex normals not being transformed, so I m4x4 them with the transposed inverse of the world transform, then normalized them. The odd part is, now those vertices will not light up when the light source is not hitting them, but the vertices that are being hit won't light up either! IE: In a cube, the front part will light up if its facing the light source, but the back part will not.

Any thoughts on that, what am I doing wrong, or if I should just let the FVF handle lighting?

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.