Shader "plugin" system?

In your ambient shader have something like:

#ifdef OPTION_04 // SSAO
vDiffuseMaterial.rgb *= 1.0-SSAO.Sample(InputSampler, i.vTex0).r;
#endif

Then you need to load a permutation of that shader with the macro "#define OPTION_04\n" prepended when your SSAO-plugin is enabled. In your config file you could specify which numbered options need to be enabled.

It has been a long time since I touched any rendering related code, but I'll try to describe what I remember from my implementation.

Each surface shader (e.g. a shader that will be applied to the surface of a 3D model) can use one or more different plugins.

Shader plugins were implemented using Cg interfaces.

So, your example above would look something like this (pseudocode since I haven't written a line of Cg for a couple of years now).

IAmbientLighting g_AmbientLighting;
sampler2D DiffuseTex;

float4 mainPS(VS_OUTPUT i)
{
  float4 diffuseColor;
  diffuseColor.rgb = tex2D(DiffuseTex, i.UV.xy).rgb;
  diffuseColor.rgb *= g_AmbientiLighting.CalcAmbientLight(i.PosWS.xyz);
  diffuseColor.a = 1.0;

  return diffuseColor;
}

The IAmbientLighting interface would look like this:

interface IAmbientLighting
{
  float3 CalcAmbientLighting(float3 posWS);
}

Your current shader would have used a constant ambient color implementation. Something like:

class ConstAmbientLight : IAmbientLighting
{
  float4 AmbientColor;

  float3 CalcAmbientLighting(float3 posWS)
  {
    return AmbientColor.rgb * AmbientColor.a;
  }
}

If you would like to change to an SSAO implementation, instead of using this class you would use:

class SSAO : IAmbientLighting
{
  sampler2D SSAOTex;
  float4 AmbientColor;
  float4x4 WSToSSMatrix;

  float3 CalcAmbientLighting(float3 posWS)
  {
    float2 screenSpacePos = TransformToSS(posWS, WSToSSMatrix);
    float ssao = tex2D(SSAOTex, screenSpacePos).r;
    return AmbientColor.rgb * AmbientColor.r * ssao;
  }

With those two interface implementations available, the renderer is responsible for selecting the correct one at run-time, based on some criteria (user prefs, GPU caps, etc.) and linking it to all the surface shaders which use an IAmbientLighting object.

The idea can be extended to other things. E.g. different kind of lights (omni, point, directional) can be implemented as interfaces of one common ILight interface.

This way you can create (e.g.) a Phong shader with or without SSAO, using one or more lights of any type.

That's the basic idea. Hope it makes some sense. If not, just say it and I'll do my best to describe it better.

Then you need to load a permutation of that shader with the macro "#define OPTION_04\n" prepended when your SSAO-plugin is enabled. In your config file you could specify which numbered options need to be enabled.

Thanks for the input, however my problem is slightly more complicated. I am well aware of shader permutations, however in my case I can't know in advance which options need to be available. Imagine someone was writing the lighting plugin, and only that; he/she wouldn't know what else someone using that plugin might need - SSAO, or even more complex global illumination models. I can't just pre-define some options to choose from, I need a way of introducing new functionality to a shader without directly modifying it.

That's the basic idea. Hope it makes some sense. If not, just say it and I'll do my best to describe it better.

Thanks, makes perfect sense, and that seems to be just what I'm looking for. A quick google-search brought me to "dynamic linkage" for HLSL 5, thats pretty much the same what you described, right? If so, is there any way to apply that to older versions of HLSL (like 3 for DX9), or would I have to use a 3rd-party shader language or unroll my own to make that happen?

Thanks, makes perfect sense, and that seems to be just what I'm looking for. A quick google-search brought me to "dynamic linkage" for HLSL 5, thats pretty much the same what you described, right? If so, is there any way to apply that to older versions of HLSL (like 3 for DX9), or would I have to use a 3rd-party shader language or unroll my own to make that happen?

The only other option I know of is to break your shaders into components written in your own meta language, usually just bits of HLSL or Cg with extra syntax to describe interfaces and what have you. Once all your plugs-in are loaded the shader components can have the HLSL or Cg extracted and pasted together to form full shaders programs which can then be compiled. Essentially this is the same as dynamic shader linkage but it's done at a higher level and by application code. It sounds like a pain to get working but it's not too bad. The primary headache (in my experience) is detecting when a permutation of components will form shader code that won't compile and spitting out an intelligent error so whoever knows what shader component is causing the problem and how to fix it. Printing the compilation error works okay for a while until the code that builds the end game shader code starts doing sophisticated things like renaming variable or other direct modifications to the pure HLSL or Cg bits.