using effect shader on RHW vertices

you don't have a pixel shader, and you are not passing the color component of your vertices into your shaders, so i would assume it would be being rendered black.

i just realized your setting the color in your vertex shader. but still, I think you need a pixel shader to specifically return that color you are making. the semantic "COLOR" is not a recognized semantic in the shaders, it is a custom semantic, which means that it is used to pass data between shaders, but the shaders themselves wouldn't know what to do with it.

try making a simple pixel shader like this:

float4 PShader(VertexOut input) : SV_Target

{

return input.Color;

}

in your technique and pass, do this:

SetVertexShader( CompileShader( vs_4_0, VShader() ) );

SetPixelShader( CompileShader( ps_4_0, PShader() ) );

also, do this:

[color=#000088]struct[color=#000000] [color=#660066]VertexOut
[color=#666600]{
[color=#000000] float4 [color=#660066]Pos[color=#000000] [color=#666600]:[color=#000000] SV_POSITION[color=#666600];
[color=#000000] float4 [color=#660066]Color[color=#000000] [color=#666600]:[color=#000000] COLOR[color=#666600];
[color=#666600]};

This will tell the rasterizer stage that Pos is the position of the vertex (notice the "SV_" prefix, this means it is a system value. system values are understood by pipeline stages, while other semantics like "COLOR" are custom, and used to pass data between programmable shaders like the vertex, geometry, and pixel shaders for example)

should it not be ?

VertexShader = compile vs_2_0 VShader();
PixelShader = compile ps_2_0 PShader();

oh yeah, if your graphics card can't handle anything above version 2, then just use version 2. that's just the shader version, and i've only ever used version 4 and 5. just use whatever works i guess

this dosent seem to be working with the SV stuff, not sure what todo now

well, what exactly is your problem. is there an error? or is it compiling but your just not seeing what you expect to see? if your not seeing what you want, what are you seeing?

What are the values of your view/projection matrices?

if your graphics card can't handle anything above version 2, then just use version 2. that's just the shader version, and i've only ever used version 4 and 5. just use whatever works i guess

this dosent seem to be working with the SV stuff, not sure what todo now

D3D9 only supports up to version 3. "The SV stuff" doesn't exist in version 3 (or has different names).
On shader model 2, it is valid to have an effect that only has a vertex shader with no pixel shader - the fixed function shading/texture-combiner functionality will be used.

here is my code


if(g_effect)
{
g_effect->SetMatrix("Projection", &matProjection);
g_effect->SetMatrix("View", &matView);
g_effect->SetMatrix("World", &(matFinale));
g_effect->Begin(NULL, NULL); // begin using the effect
g_effect->BeginPass(0); // begin the pass

d3ddev->SetFVF(CUSTOMFVFRHW);
d3ddev->SetStreamSource(0, v_buffer, 0, sizeof(CUSTOMVERTEXRHW));
d3ddev->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 1);

g_effect->EndPass(); // end the pass
g_effect->End(); // end the effect
}
else
{
//d3ddev->SetFVF(CUSTOMFVF);
//d3ddev->SetStreamSource(0, v_buffer, 0, sizeof(CUSTOMVERTEX));
//d3ddev->DrawPrimitive(D3DPT_TRIANGLELIST, 0, 1);
}



shader.fx
float4x4 World;
float4x4 View;
float4x4 Projection;
float BoundingBoxColor; //float passed by param...
bool setmode;

struct VertexOut
{
float4 Pos : POSITION;
float4 Color : COLOR;
};

VertexOut VShader(float4 Pos : POSITION)
{
VertexOut Vert = (VertexOut)0;
float4x4 Transform;


Transform = mul(World, View);
Transform = mul(Transform, Projection);
Vert.Pos = mul(Pos, Transform);
Vert.Color = float4(1, 0, 0, 0);
return Vert;
}

float4 PShader(VertexOut input) : COLOR
{
return input.Color;
}

technique FirstTechnique
{
pass FirstPass
{
//Lighting = FALSE;
//ZEnable = TRUE;
//FillMode = WireFrame;
//FillMode = Solid;

VertexShader = compile vs_2_0 VShader();
PixelShader = compile ps_2_0 PShader();

}
}


its just rendering normal

It's been a long time since I've used FVF codes, but I'm reasonably certain that using D3DFVF_XYZRHW will bypass the vertex shader entirely (D3DFVF_XYZRHW maps to D3DUSAGE_POSITIONT which specifies that vertex processing is skipped). In particular it makes no sense to be applying transforms to your position as the purpose of XYZRHW is to output untransformed vertex positions. So you should switch to D3DFVF_XYZ and seriously consider switching to vertex declarations too (there's a handy D3DXDeclaratorFromFVF which will help in the initial transition).

What mhagain said. D3DFMV_XYZRHW means "Here is a vertex is screen space, don't run the vertex shader". And I'd strongly second getting away from FVF codes - Declarations are much easier to map with vertex shaders, and FVF codes don't even exist in DX10+.