Skip to main content
GameDev.net gamedev.net
🔒 Locked

Shader Organization

Started by AvengerDr Aug 24, 2008 at 7:31 PM 6 replies 2.7k views
Original Post
AvengerDr
AvengerDr
I've been thinking about how to organize my shaders: even in the simple case of a specular lighting shader there may be several techniques.. Pure specular lighting, specular with a diffuse map, specular with texture and normal map, pure specular with shadows and no textures and so on. One could theoretically treat each particular combination as an entirely different shader, but for the sake of code duplication (and readability) maybe it's better not. Now, in a "real" game, the choice of which technique to use could be dictated by which option the player has chosen in the "option screen". For example if the player chose "no shadows" then only techniques with no shadows will have to be used. Similarly if he chooses "high shadows quality" then the engine should employ VSM (I'm moving my first step in the shadows domain (!), i hope they can be considered high quality enough) with an high resolution enough shadow map. Now, how should the engine handle all of this? I can think of two solutions, but since I'm not that experienced in this field I'm interested in hearing what you might have to say. The simplest solution would simply be to keep an hard-coded "map" of techniques linked to particular combinations of options. Like: if OptionA is true and OptionB is true then use TechniqueX for EffectY. But this sounds rather unelegant. The other solution could be to dynamically build the technique name from the option chosen. If SpecularBump is eneabled and shadows are disabled then the resulting technique may be called "SpecularBumpUnshadowed".. Obviously one would have to follow the naming scheme in the shaders, but it seems to be that this scheme wouldn't break as often as the first one. Anyway, are there better solutions? Another questions that I added concerned the use of #include directives in shaders. Again, for the sake of code duplication and readability, I was considering moving light calculations and other common variables and functions into included files. Since I plan to use EffectPool (my current implementation does not yet take advantage of it), many matrices would be common to many effects. I am already using the same names across all of my shaders. The question that I have is the following. Suppose that I have the default texture declaration for shadow maps into one of the included file, along with its accompanying sampler declaration. Suppose now that I'm using for an effect a technique that does not use shadow maps. Would that texture declaration in the compiled HLSL file waste some memory? Does some memory allocation process occur when the compiler finds one of those declaration? Or maybe does it only occur when the value is actually assigned? Or better yet, the compiler is smart enough to check whether that variable will even be utilized at all? I think it's the latter, because while debugging with PIX I have noticed that some statements seem to be only executed when they are needed even if they are the first lines in the shader. I think this is maybe a non issue, but one never knows.. Thanks in advance :)
fitfool
fitfool
Maybe you should handle this in the shader itself. Have a single pixel shader and a single vertex shader, have it take a few uniform booleans(uniform bool useShadows, uniform bool hasSpecularLight, etc.) and have a different technique for each combination.

(pseudocode)
technique shadowsNoSpecular{     vertexShader = vertMain( true, false );     pixelShader = pixMain( true );//toon shading, no p}


Or have a pass for each needed effect, a pass for shadows, a pass for specular lighting etc. Hope this helps :D
hh10k
hh10k
Why offer so many options to the end user? Finer control for features is great for advanced users, but if you're writing an engine from scratch a more reliable solution would be to come up with a set of profiles for hardware levels. It also makes the artist's job easier when there are fewer permutations of things to make look good.
MJP
MJP
Yeah I'd probably recommend going with the "seperate techniques that use uniform int's and bool's" if you're not going for a huge-scale engine. I mean it's great if you can get a full shader pipeline going that can assemble shader fragments to compile 8000 different shader permutations and still be able to debug your shaders, but it's a pretty significant task that would probably require creating a few offline tools.

One thing that might help you keep your number of permutations down is to use deferred shadow maps. The idea instead of sampling your shadow map and performing the comparison/PCF as your rendering the geometry, you do it all in a full-screen (or portion of the screen, for non-global lights) pass where you determine the position of a fragment by reconstructing it from a depth buffer. Then you render the result of the shadow comparison out to a screen-sized texture, and sample that when you're rendering your geometry and calculating the lighting. It's nice from a logistics point of view, since your forward lighting pass doesn't know how the shadow occlusion is calculated and doesn't care: it could be with stencils, shadow maps, VSM, whatever. It just samples the shadow occlusion value and uses it. You also don't even need to switch shadows on and off in your lighting shader, you can just send it a 1x1 white texture and there will be no shadows. It's also pretty nice performance-wise, since you don't have to use complex geometry with small triangles (which are inefficient, since they don't always cover all 4 pixels of a quad).
AvengerDr
AvengerDr
Thanks for your input. I agree with your suggestion, it seems the most sensible. I was asking that because, by modifying a specular lighting shader in order to display shadows.. I lost the possibility of using the specular shader alone without shadows. Unless obviously I make a new technique that doesn't calculate shadows.

This uniform modifier means that it's some sort of constant parameter? I mean, once compiled, the branching will be eliminated?
MJP
MJP
Yup, uniform means that the value is held constant through the lifetime of a shader. So if you set the value in a pass or technique declaration, the FX compiler will optimize out any branches/loops/operations that use the value. If you have several techniques or passes that use different values for the parameter, you'll get several different compiled versions of the shader.
AvengerDr
AvengerDr
I've been refactoring my shader code so that I could make use of different techniques via uniform bool(s). I've realized that I'm slowly creating a "uber-shader". Previously I had several smaller shaders.. You see, I had a "Diffuse.fx" shader for Phong-diffuse only, a "specular.fx" for Phong w/specular highlights and so on.

Now I have only a "Phong.fx" material, and moved common variables into an included "lighting.fxh" file. So I made several techniques that enable/disable certain parts of the lighting formula. This also makes more sense from a 3dstudio-ish perspective. Where you have to choose which shader (among Phong, Blinn, Oren-Nayar, Anisotropic, etc...) to use. This also means that now that I just have Phong lighting, I should implement the others :D Is the book "Advanced lighting and materials with shader" still good? Not that lighting equations would change but since the book is from 04 maybe there's something better around?

Anyway, I've some questions. I wrote two techniques, one for diffuse only that doesn't compute the specular component, and one that does it instead. Now if I wanted to throw normal mapping in the Phong-lighting (uber)shader I can't use the basic VS-In struct anymore. I should use another that passes the tangents and binormals too.

So I had to duplicate the Vertex Shader and Pixel Shaders function. Although I do the Phong equation calculations into a "PhongLighting(....)" function (that is called by both shader variations), I still had to duplicate them. Is there a way around? An elegant solution! I could play around with #ifdef and #ifndef, but then things will start getting ugly..

Another question that I had (maybe it is more relevant to the DX-forum) regards the different VS/PS profiles. If a user's card supports SM3 it is always preferable to compile for SM3 even if a particular shader could still be compiled for SM2.0? I'd say yes, because I noticed that in rendermonkey compiling a shader for SM2 yielded 55 instructions. While compiling for SM3 yielded 53 instructions (!).

Last question.. More complicated :D Let's say that I have 2 object that share the same material. But they have different properties; maybe one uses a blue diffuse only lighting and the other specular textured lighting. In my engine, materials are individual to each object but they do share the same Effect.FromFile(...) reference if they use the same effect.

This mean that they will both share the same "Phong.fx" effect object. To make it work correctly I should switch techniques back and forth, which I assume it is always better to avoid.

I'm also using an "effect cache". When a material asks for an effect, the effect cache checks whether it already contains this effect. Now, hoping that an effect compiled with Technique A would return a different hashcode than an effect compiled with Technique B and 2 effects compiled both with Technique A would return the same hashcode, I could use that value as the key. So in the cache I'd store each shader permutation but multiple materials wouldn't have to change technique every frame because they'll have a completely different object. Right? If the internal hashcode does not work as expected I can always generate this key based on the above premises. Right now I'm using the filename as the key.

This assumes though that If I call Effect.FromFile 10 time on the same effect file, I get 10 different effect objects that aren't related between each other in any way even if i use the same technique.

Has anyone tried a similar approach?

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.