hardware branching

It does the test on groups of pixels (typically a 2x2 block). If it can't skip the code for all, it skips for none.

Also in my experience, having dynamic branches in code like that severely reduces performance (I measured a 100 cycle penalty for branching, to the extent that it was a lot faster not to branch).

The GPU will evaluate the conditional for the entire block of pixels it's currently processing (this is something like 64 or 128, maybe larger or smaller depending on hardware). If the conditional evaluates to the same result for all of them, then only one branch is executed. Otherwise both branches are executed.

That particular conditional has very good coherency for interpolated normals so it's very likely that an entire block will pass or fail the test. Normals maps aren't going to be so coherent because they're usually used in order to simulate rough, high-detail surfaces. I'm not sure if it's possible to modify the block size using standard OpenGL or D3D, but I know that with CUDA you can mess around with it.

The GPU multiprocessors work in SIMD, which forces each individual processor to either do the exact same operation as the others (for its pixel) or do nothing on a given clock cycle. So, if a branch happens and one of the processors in the multiprocessor follows it but the others don't, then that processor will be executing its instructions while the others will be doing nothing, until the control flow reunites. So, while some processors will not execute the operation, they will still wait, doing nothing, for the duration of the operation.

In short, on the GPU, if (A) {B} does not mean 'if A then execute B' but rather, 'execute B and, if A, apply the results'.

The exception here is when all the processors skip the conditional. When that's the case, they don't have to wait for one of them to be done with the conditional body, and may move on directly. Where Zipster made a mistake is in equating this with a block—this is incorrect. What is relevant is the warp size, a warp being the number of pixels which are evaluated in SIMD (and are thus forced to run together the same instructions). The CUDA developer documentation explains that warp size for 1.0 hardware (GeForce 8800, Quadro FX 4200) is fixed at 32 because of cycle length and processor count, so any 32-pixel group skipping the conditional will not execute it, even though the next 32-pixel group doesn't. However, these video cards will execute several warps in parallel (typically for a simple pixel shader, four warps per multiprocessor, sixteen multiprocessors per card).