🔒 Locked

Managing multiple GLSL programs

Started by Aikavanak Jul 2, 2008 at 4:08 AM 12 replies 8.7k views

Original Post

Jul 2, 2008 at 4:08 AM

Hello, I have just recently started working with GLSL, and would like to create multiple shaders for each of the effects in my engine. I have one for per-pixel lighting, and eventually I will have more for shadowing, skinned animation, normal mapping, and more. Of course I would like to allow the users to enable and disable certain features (shaders) so they can get the best performance. But what's the best way to manage this? I have thought of two possible solutions, and am wondering which is better, or if there's another, even better option. I haven't seen many threads on this issue. Both of my ideas would have each "effect" in it's own function and it's own file. But since a program can only have one main function, how should I determine which other functions to call? 1) Create a main() shader for every possible combination of enabled/disabled effects, and load the correct one when the user changes the settings. I'd rather not do this, as the number of main() functions could get out of hand quickly. 2) Have just one large main() shader that takes uniforms to determine which features are enabled, and use many if/else blocks to call the required effect functions (from other files). I'm leaning towards #2, but is there a better option? I know uniforms cause performance problems if they're changed often, but would there be any performance issues with my intended use (only changing when the user updates graphics settings)? Thanks!

MARS_999

Jul 2, 2008 at 4:58 AM

You are better off using the many shaders approach. Meaning you code each shader tailored for what you want to do. So if you want the best performance shadowmap shader then you code it and name it shadowmapFast.vsp shadowmapFast.fsp vs. shadowmapQuality.fsp shadowmapQuality.vsp, hope that helps out. BTW I would make a resource manager for all of your shaders. As you may end up with a lot of duplicates eventually.

And if the user has selected Fast for a graphical setting you just use those shaders or if they selected Quality then call those shaders.

dmatter

Jul 2, 2008 at 6:53 AM

Quote:
Original post by Aikavanak
I'm leaning towards #2, but is there a better option? I know uniforms cause performance problems if they're changed often, but would there be any performance issues with my intended use (only changing when the user updates graphics settings)?

Branching in shaders is a relatively new feature and is still rather costly, there would be a constant performance hit from executing the shader.

swiftcoder

Jul 2, 2008 at 6:58 AM

Quote:
Original post by dmatter
Quote:
Original post by Aikavanak
I'm leaning towards #2, but is there a better option? I know uniforms cause performance problems if they're changed often, but would there be any performance issues with my intended use (only changing when the user updates graphics settings)?

Branching in shaders is a relatively new feature and is still rather costly, there would be a constant performance hit from executing the shader.

Here he would probably only be relying on the compiler optimising away the disabled code - but using the preprocessor's #define/#ifdef/#endif is a more reliable way to do this, I think.

trippytarka

Jul 2, 2008 at 8:06 AM

Option 3) Produce the code for the main function at runtime according to the user settings.

ZeroSum

Jul 2, 2008 at 8:18 AM

Quote:
Original post by trippytarka
Option 3) Produce the code for the main function at runtime according to the user settings.

Out of interest is this what the Battlefield games (2 and 2142) do? After changing video settings the next time you load a map the game spends a little while "Optimizing Shaders", I just never figured out what it was exactly doing.

Ekast

Jul 2, 2008 at 9:05 AM

Quote:
Original post by ZeroSum
Out of interest is this what the Battlefield games (2 and 2142) do? After changing video settings the next time you load a map the game spends a little while "Optimizing Shaders", I just never figured out what it was exactly doing.

I think that's what battlefield does. After changing different settings like detail and other shader based things it could be possible that Battlefield "recompiles" all the relevnt shaders before starting the game.

I'm currently working on a similar system to combine shaders at runtime or beeing more precisely, after creating the SG for the very first time. Before entering the rendering-system the entire SG is beeing traversed and checked for all different shader-combinations.
Let's say the topmost SG-node contains a shader node that says: "All objects below have to use at least one texture". A object below the root node might have another request of using normal-mapping. Then the shader handling normal-mapping is beeing attached to that node. When it now comes to rendering the renderer knows that this objects has to be drawn using one texture and an additional normal-mapping effect. The advantage of such a system is that you don't have to write the "SuperShader" which contains a lot of if/else branches but you just have to write "little" shaders containing one specific feature like texturing/reflection/normal-mapping...
As mentioned above performing the preprocessing before entering the renderer allows you to find out all possible combinations of shaders, which now have to be "glued" together. This however unfortunately needs some thinking about how to combine all shaders into one which is not that easy.

Before using the above system I wrote a shader for each different combination but as you might imagine this works for a small amount of effects but gets quite messy when handling a whole bunch of shaders.

Hope that helps

Ralph

[Edited by - Ekast on July 2, 2008 2:05:18 PM]

LoneDwarf

Jul 2, 2008 at 12:27 PM

I haven't hit this part in my project yet, but what I expected to do was use the preprocessor to conditionally compile the shaders. I am pretty sure glShaderSource was setup to be used like this since it has the option to pass an array of strings. If you don't like using the preprocessor then maybe you could break the code into functions and then conditionally assemble them with the linker, I do this now. I am not sure how optimal this code it though.

So I would have your user preferences mapped to defines that get added as strings before the actual shader source. It's the same as managing different targets for a C/C++ project. GLSL has a preprocessor, compiler and linker so you can solve it the same way.

I would never code these functions by hand BTW. Back in the day when shaders were new most people used a C preprocessor on thier code to generate different versions for stuff like number of lights or simple component like assembly. I did this for the Xbox and also on the PS2 (vu code).

I also wouldn't waste a uniform on this. It seems like you are defeating the whole point by expecting your fallback to support branching which it likely doesn't. Also it's likely low on uniforms too.

Aikavanak

Jul 2, 2008 at 3:11 PM

Thanks for all the replies!

I had a feeling there were better options out there.

I like trippytarka's idea of generating the main() function at run-time. It seems pretty easy too, sine glShaderSource does take an array of strings.

I can also see how the preprocessor could work, but wouldn't this also require setting the #defines at run-time?

Looks like the two new options are:

1) Don't use the preprocessor. Just store strings that call the different shaders and combine them into a main() function when settings are changed.

2) Write one, big, static main() function with lots if #ifdef directives that activate features, and store strings of #define directives that are added to the beginning of the shader when settings are changed.

Is one of these better/more efficient than the other? It seems like the final compiled and linked shader would the same in both cases.

LoneDwarf

Jul 2, 2008 at 3:49 PM

Quote:
Original post by Aikavanak
Thanks for all the replies!

I had a feeling there were better options out there.

I like trippytarka's idea of generating the main() function at run-time. It seems pretty easy too, sine glShaderSource does take an array of strings.

I can also see how the preprocessor could work, but wouldn't this also require setting the #defines at run-time?

Looks like the two new options are:

1) Don't use the preprocessor. Just store strings that call the different shaders and combine them into a main() function when settings are changed.

2) Write one, big, static main() function with lots if #ifdef directives that activate features, and store strings of #define directives that are added to the beginning of the shader when settings are changed.

Is one of these better/more efficient than the other? It seems like the final compiled and linked shader would the same in both cases.

Quick contrived example:

void main(){	    for( int b = 0; b < NUM_BONES; ++b )    {#if defined(SHADOW_GEN)        // whatever you might need.#else        // fallback here I guess#endf#if defined(PER_PIXEL_LIGHTING)        // whatever you might need.#else        // fallback here I guess#endf    }}

Then use it like:

    const GLchar* theCode = // Load the file from above somehow.//    std::vector< const GLchar* >    vecStrings;//    const GLchar c_Defines[] =     {         "#define DEBUG\n",        "#define SHADOW_GEN\n",        "#define PER_PIXEL_LIGHTING\n",    };//    char	defNumBones[256];    sprintf( defNumBones, "#define NUM_BONES %d\n", prefs.numBones )    vecStrings.push_back( defNumBones );//#if defined(DEBUG)    vecStrings.push_back( c_Defines[0] );#endif    if ( prefs.bShadowGen == true )        vecStrings.push_back( c_Defines[1] );//    if ( prefs.bPPLighting == true )        vecStrings.push_back( c_Defines[2] );//    vecStrings.push_back( theCode  );//    int vs = glCreateShader( GL_VERTEX_SHADER );//    glShaderSource( vs, vecStrings.size(), &vecStrings[0], NULL );    glCompileShader( vs );

Aikavanak

Jul 3, 2008 at 2:39 PM

Hey,

Thanks for the example Tesshu, that's exactly what I was talking about as option #2 in my last post. But why not just build the whole main function dynamically (option #1)? Since the different detail level shaders will be in their own functions, couldn't I just piece together those function calls into a main() function, and avoid the preprocessor altogether?

I would appreciate it if anyone could give some pros/cons of each of these ideas.

Thanks!

kidmosey

Jul 11, 2008 at 10:14 AM

It seems the limit to the number of shader instructions and how OpenGL compiles programs could be a factor.

If all functions are linked into the final program regardless of whether they are called from main(), then you are better off using the preprocessor to cut out sections of code that you will not be using.

McFury

Jul 23, 2008 at 10:41 AM

Hi,

Wow this post was a great read thanks for discussing it.

I have a question though (and I hope I understand the points raised). If using the pre-processor approach and if the shaders being applied were not just constant shaders (i.e. lighting,shadows) but also dynamic ones (say a character in a game pulls out a sword that has a shader effect applied to it).
Such an effect wouldnt have been enabled with the others, but would need to be applied during runtime.

How would this type of situation be handled?

Thank you!

V-man

Jul 23, 2008 at 12:36 PM

Quote:
Original post by McFury
Hi,

Wow this post was a great read thanks for discussing it.

I have a question though (and I hope I understand the points raised). If using the pre-processor approach and if the shaders being applied were not just constant shaders (i.e. lighting,shadows) but also dynamic ones (say a character in a game pulls out a sword that has a shader effect applied to it).
Such an effect wouldnt have been enabled with the others, but would need to be applied during runtime.

How would this type of situation be handled?

Thank you!

All the shaders that you will ever need during the game should be compiled and ready for use. If you compile and link in the middle of game play, you might see a sudden pause for 1/2 a second or something.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.