There is an approach in all samples I saw:

1. Take an fx file in runtime.
2. Add macros from config
3. In runtime on user machine compile shaders with providing to FX compiler #defines
4. Load and run the compiled code.

I can see several things I would like to avoid:

1. FX format is deprecated
2. Compilation time at user’s machine is not free
3. Distribute HLSL (FX) code to user machine is not a good thing, because it can be quite fast reverse engineered.
4. I am C++ guy, so #defines bothers me because of a lot of bugs (it is C legacy).
5. Code become not elegant with #defines

Question 1:

When I compile HLSL code to .co object on my machine, and distribute the binary to user’s machine, is it less efficient than compilation is taking place on a user machine?

Or compilers are the same?

Question 2:

Is there a way to write C++ - style shaders without #defines?

For example, is it possible to make code like this without preprocessor?

(Target platform: Windows10+, SM 5+, no OpenGL, no consoles)

#if MSAA_
   Texture2DMS<float> DepthMap : register(t0);
#else
   Texture2D<float> DepthMap : register(t0);
#endif

#ifndef MSAASamples_
   #define MSAASamples_ 1
#endif

#if MSAA_
   uint numSamples;
   DepthMap.GetDimensions(textureSize.x, textureSize.y, numSamples);
#else
   DepthMap.GetDimensions(textureSize.x, textureSize.y);
#endif

Thanks as always

I don't think so, maybe shader model 6 will fix this. Of course you could create some higher level tool that generates hlsl code but that's just really moving the problem. You can pass in defines when you compile the shader but again, that's just kind of moving the problem as well.

Supposedly we're getting templates, virtual functions, constructors, all sorts of nonsense sometime soon.

1) Pre-compiling on your PC / late-compiling on the user's PC is the same. IMHO it's better to pre-compile and ship binary shaders to improve loading times.

2) You can reduce the amount of #defines required by using functions and branches.
Branching is resolved at compile time if the value is known at compile time - e.g. either DoA() or DoB() will get compiled here, and the other will be removed from the shader:

#if MSAA_
const static int frob = 42;
#else
const static int frob = 12;
#endif

void test()
{
  if( frob > 20 )
    DoA();
  else
    DoB();
}

Function overloading also lets you take different paths here without repeating the #if again:

#if MSAA_
   Texture2DMS<float> DepthMap : register(t0);
#else
   Texture2D<float> DepthMap : register(t0);
#endif

void GetDimensions( Texture2D<float> t, out uint2 textureSize, out uint numSamples)
{
  numSamples = 1;
  t.GetDimensions(textureSize.x, textureSize.y);
}
void GetDimensions( Texture2DMS<float> t, out uint2 textureSize, out uint numSamples)
{
   t.GetDimensions(textureSize.x, textureSize.y, numSamples);
}

void test()
{
  uint2 textureSize;
  uint numSamples;
  GetDimensions( DepthMap, textureSize, numSamples );

  if( numSamples <= 1 )//!!
  {
    DoStuff();
  }
}

The line marked with //!! is interesting -- when MSAA_ is true, this will actually be a real runtime branch, but when MSAA_ is false, it will be resolved at compile time!
You could fix this issue with the code below, which would let the compiler always be able to resolve the "if(numSamples<=1)" branch at compile time :wink:

void GetDimensions( Texture2DMS<float> t, out uint2 textureSize, out uint numSamples)
{
  t.GetDimensions(textureSize.x, textureSize.y, numSamples);
  numSamples = max(2,numSamples);//assumption: Texture2DMS can't have 0/1 samples...
}

SM5 also introduced dynamic linking and interfaces... Though I prefer to do things the old way :P

I am C++ guy, so #defines bothers me because of a lot of bugs (it is C legacy).

While you may have some valid concerns regarding macros, this is not one of them. They were introduced to create “inlined” functions that C lacked. If you believe that this form of their use is legacy now that C++ is finally out, then you should only be bothered when macros are used as inline functions in C++.
Otherwise, they still play a vital role in C++ for cross-platform support, metrics, etc. There is absolutely nothing “lot of bugs” about:

#if MSAA_
   Texture2DMS<float> DepthMap : register(t0);
#else
   Texture2D<float> DepthMap : register(t0);
#endif

When I compile HLSL code to .co object on my machine, and distribute the binary to user’s machine, is it less efficient than compilation is taking place on a user machine?
Or compilers are the same?

How fast the compiler compiles depends on their machine.
How fast the code is depends on the version of their compiler. There shouldn’t to much variance here—versions are likely to be the same, or theirs will be better over time.

Is there a way to write C++ - style shaders without #defines?

The same way you would write C++ without #define’s: Slowly and painfully. Maybe that means copy-paste in a bunch of places. Maybe that means writing a whole bunch of files with only slight differences in them. Maybe it means creating wise inlined functions. Whatever you do in C++ would work in HLSL.

Is it possible to make code like this without preprocessor?

How would you do it in any other language? I expect you would have to create a bunch of files that are basically the same but with the small changes your macros make. Keeping them all up-to-date as you upgrade your shaders is unreasonable, and creating a mess of files just to avoid safe usage of macros is even more unreasonable.


Solution: Don’t be obsessive-compulsive about safe usage of macros. Just use them as they are to be used and make your life easier.

…than compilation is taking place on a user machine?

Compilation never takes place on the user’s machine. Your tools compile every possible permutation of the shaders and they are distributed with the game. If there are too many permutations, then your tool needs to wisely select all permutations that will actually be used. This can be done by you actually setting up the permutations to compile, and/or by running the game with a debug feature enabled that keeps track of every permutation as they are created on your end.


L. Spiro

Though I prefer to do things the old way :P

Can I ask: why?

Though I prefer to do things the old way :P

Can I ask: why?

Because KISS.
Disciplined use of macros to select different permutations (and then using normal function calls for most of the shader) is IMHO far simpler than this interface/linking mechanism.
The interface/linking mechanism is also fairly specific to D3D11, so a shader system designed around it wouldn't port to other platforms easily.

p.s. I edited my earlier post...

…than compilation is taking place on a user machine?

Compilation never takes place on the user’s machine. Your tools compile every possible permutation of the shaders and they are distributed with the game. If there are too many permutations, then your tool needs to wisely select all permutations that will actually be used. This can be done by you actually setting up the permutations to compile, and/or by running the game with a debug feature enabled that keeps track of every permutation as they are created on your end.

I never heard about permutation tools.

Right now I use default properties of HLSL file inside VS and change only few of them:

- Shader type (VS/PS/GS/CS)

- Entry point name (if I name a function different than main())

- Till now I did not use "Preprocessor Definitions" property at all.

Also I have vs.props file will common HLSL-related definitions that apply to all HLSL-files:

    <FxCompile>
      <ObjectFileOutput>$(OutDir)Shaders\%(Filename)</ObjectFileOutput>
      <ShaderModel>5.0</ShaderModel>
      <AdditionalIncludeDirectories>$(SolutionDir)Libs\MGL</AdditionalIncludeDirectories>
    </FxCompile>

So, VS compiles hlsl file by data, contains in vcxproj file.

That allows me have only 1 compiled shader per hlsl-file per Debug/Release.

Can anyone know:

1. Is it possible to have multiple outputs for 1 HLSL file using native Visual Studio functional (via properties of HLSL file)?

2. Is there a standard tool to compile HLSL code with all permutaions?

3. Or do I need to create it by myself?

3.1. Besides different macro permutations, what other feature can be added to the tool (I just learning to write games, so any information is welcome)?

Thanks.

I don't think I've ever seen an engine than makes use of dynamic linking.

2. Is there a standard tool to compile HLSL code with all permutaions?

No.

3. Or do I need to create it by myself?

Yes.

3.1. Besides different macro permutations, what other feature can be added to the tool (I just learning to write games, so any information is welcome)?

Whatever you want.
The tool will just call FCX.exe, which is all Microsoft® Visual Studio® is doing.
Your tool could even be just a .BAT file with a bunch of calls with different permutations, perhaps an input parameter to tell it debug or release, etc.
And you could generate all these unique permutations by playing the game and having the game log them all (generating the .BAT file for you).
But you have to be sure not to miss any.


L. Spiro

L Spiro mentioned this, but I want to emphazise: The macros that are discouraged are the ones being used as an inline function. That's because they don't behave the way you would expect.

For example:

#define MIN( x, y ) ( x < y ) ? x : y;
float a = 1;
float b = 1;

float r = MIN( a++, b );
//now r = 1; a = 2;

That's because it will translate to:

float a = 0;
float b = 1;

float r = (a++ < b) ? a++ : b;

which is probably not what you intended.

However using #ifdef #else #endif is perfectly fine, other than becoming less pretty to the eye, or going out of hand if not being disciplined; which is the reason some people (i.e. me) prefer to run the code through your own preprocessor to ease some stuff.