Skip to main content
GameDev.net gamedev.net
🔒 Locked

[HLSL] incorect result from modulus on x64

Started by ender_341 Aug 21, 2009 at 7:12 PM 1 replies 1k views
Original Post
ender_341
ender_341
when compiling a shader on an x64 build of my code the following code generates the wrong values, but on x86 build it works fine: sm3.0, march 2009 SDK

uniform extern int    character;
uniform extern float3 position;

uniform extern float2 texSize;
uniform extern float2 charSize;

uniform extern float4x4 wvpMatrix;

struct VSOut{
  float4 hposition : POSITION0;
  float2 texCoord  : TEXCOORD0;
};

VSOut FontVS( float3 pos : POSITION0, float2 texCoord : TEXCOORD0 )
{
  VSOut vsOut;
  vsOut.hposition = mul( float4( position.xy + ( position.z * pos.xy ), 0.0f, 1.0f ), wvpMatrix );

  float2 charloc = float2( character % 15, character / 15 );

  vsOut.texCoord = texSize * charSize * ( charloc + texCoord );

  return vsOut;
}

I've narrowed it down to the line float2 charloc = float2( character % 15, character / 15 ); If I upload the character % 15 from the CPU the code runs fine. Is there any known bugs on the X64 HLSL compiler or is there some other reason this would be screwing up?
-Matt S.
DieterVW
DieterVW
What flags are you using to compile your shader? The March 2009 SDK produces no asm differences between the x86 and the x64 compilers when compiling your file and using no flags. Are you seeing asm differences? Are you using REF or hardware to run your shader?
ender_341
ender_341
I'm using no flags to compile it, and I get the same output for REF and hardware.

Checking the asm I'm not getting any differences.

Using pix to check the values I'm sending the shader show that its getting sent the same values too.
-Matt S.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.