Shader optimization

1) Are dependent texture reads a real problem anymore on the newer cards? (I never really understood why they were so slow on the old cards other than maybe stalling the pipeline due to poor/non-existent thread scheduling??)

2) I remember reading somewhere that I should issue texture lookups asap so that there is work to do while we wait, is this true?[/quote]there are several reasons for this. some hardware has a special path where texture reads based on interpolators are scheduled before the shader is even executed, a texture read will cost nearly nothing in this case. Another reason is that pixel shaders are like a reduction, they gather a lot of math and data to output usually one float4 value. the later in a pixelshader you issue a texture read, the less the chance is the compiler can insert independent alu instructions before that texture fetch is used and that might rather stall the pipe. (although the newest hardware is doing its best to hide that by using a lot of threads).

3) I also remember read about how the newer cards aren't really 4-way SIMD processors (as in operate on float4's) andthat really they execute four instructions on four pieces of data in one clock cycle. Something about how NVIDIA analyzed shader usage and found that typical operations are not float4s but rather on floats. Does that mean I should not try and pack data into float4's in the shader? I'm not talking about packing data into a vertex attribute, I mean temp registers in the shader. Speaking of temp regs...[/quote]that really depends on hardware. you are right, nvidia GPUs are scalar nowadays, some vector optimizations like




lumincance=dot(color,float4(1.f));



might rather make your shader slower than faster. Although ATI also does not have a dot4 anymore, those are still vector4 and vector4+1 GPUs (the newer tend to be float4) and if you vectorize your code by hand it might be better than what the compiler creates. So it's really up to the target hardware, what you do and sadly, optimizing for one usually hurts the other one. (as far as my sample goes, nvidia driver detect that case and reorder it to 3 add, it's just an example).

4) Now that everything is unified architecture I read that I should keep GPR usage to a minimum such that more threads can be executing in parallel. This seems to conflict with 3), although I would think 4) would outweigh 3).[/quote]it's not since unified architectures that GPR usage defines a lot the final performance outcome. GeforceFX (geforce 5) really suffered if you used more than 4 registers. So keeping register usage low is usually a good thing, but that's up to the compiler in most cases. on ATI those seem to be SIMD organized, so using vectors might actually lower the GPR pressure, but yes, on NVIdia, if they don't detect what you're intending to do, it might cost you something.

5) I had a guys tell me once that every keyword in hlsl is takes 1 clock cycle (ex: dot( N, L) is 1 cycle, exp( X ) is 1 cycle, pow( X, Y), clamp(), max(), sin(), atan() etc). Is this true? He's a pretty senior guy but not really up on the new tech. I'm skeptical. I wouldn't even be so sure that one assembly instruction is equal to 1 cycle since different GPU's will decode the general assembly into something else, right? But I could be wrong (which is why I'm asking you)[/quote]on microsoft hlsl intrinsic documentation, you can read how many "slots" instructions cost, e.g. out of my head "sign" costs 3 slots. On the other side, if the GPU detects you want to get the sign, it might cost you 1 instruction or in some cases even be for free if the compiler (I mean the one inside the driver, not fxc) figures out that in combination your "sign" can substitute e.g some redudant mov. if you want to see some performance metrics, try FXComposer, it has some cycle output for shaders (at least it was the case when I tried it some years ago).

6) I was in an interview once and a graphics guy told me that sometimes interpolators can be a bottleneck in the pipeline. I can understand interpolators (color) causing some precision issues, but speed!? How? I believe interpolators are separate hardware units so maybe they can't really handle a full load when all of the vertex attributes need interpolating. (just guessing)[/quote]interpolators are not purely seperate hardware anymore, it makes nowadays no sense to have special purpose hardware that would idle most of the time (you might have 16 interpolator accesses in a shader that has 200instructions, that's 184 idle loops), that's why shader units at least assist interpolator hardware and it's all not for free. if you have a shader e.g. to blend 8 terrain layers and all you do per pixel is to read 10 textures and blend them, interpolators might be the limit. That's also the case if you use tesselation with a lot of tiny polygones, just providing the interpolators for every micro triangle that covers 1pixel in average, might make the whole rendering interpolator bound.

I'm deeply interested in these kind of things to keep in mind when writing my shaders (or interviews). I understand some of these will depend on hardware, but I'm not content with that. I want to understand WHY something is the way it is. I would much rather hear something like "Well on the PS3 it will do X, on the 360 it will do Y, and on ATI it will do Z vs NVIDIA it will do W. Documentation or an article would be the best so that I can read about these things myself.
[/quote]all hardware is designed for some typical work load, if you move away from what is expected, you will be limited by some particular part that was not designed for that purpose. In typical rendering, the RSX of the PS3 is a well designed hardware, if you'd double some part of it, you would not gain any performance, as you'd either be limited by the previous stage that was not designed to provide as much data as is needed now, by the next stage, that was not designed to consume as much data as you produce now or you'd simply lower the whole architecture performance (e.g. lower timming, less MHz) due to too much transitors.

To always accept this is a key, I see too many times people start optimizing something by assuming "hey, this is expensive, I read this somewhere, I have to optimize it" and later on they ask "just why is it not faster". You need to understand what each stage really does and how you can validate if that's a limit. e.g. if you want to figure out if you are bandwidth limited, reduce the texture size to 1x1. if you figure out, that's not giving anything, then all optimization in that directions won't help. it's like the most basic thing to first profile and find the issue and then optimize, but somehow most people avoid it and waste time.


Once you found the bottleneck, it's quite easy to find optimization suggestions.

6) If interpolators are a bottleneck in the pipeline than it's a very strange program indeed. The real pipeline bottlenecks are going to be shader complexity and fillrate; interpolators are small fish compared to this and it sounds like another dubious micro-optimization.

[color=#1C2837][size=2]I've seen this as a problem on X360 and PS3 shaders to the point where we had to pack values into different 'slots' in order to reduce our interpolator pressure. It does tend to show up more on short/already optimised shaders however.[/quote]
How do you even come to the conclusion that the interpolators are the bottleneck? Reduce the number of interpolated attributes and see if it's faster, maybe?