Deferred Shading - Deriving normal.z

Assuming that all the faces are front facing in your shading pass (ie. front facing normals/backface culling) you can assume the sign of the z component if you store them in viewspace

z = sqrt(1 - x^2 - y^2)

Hello Hogwash!
Thanks for your quick reply!

Yeah i was, and still am, a little confused about this. But it seems a normals z value can still be negative.

The first reason that comes to mind is normal mapping. A surface can still face you, but the normal per pixel is adjusted in such a way that z can be negative.

The second reason i believe has something to do with projection. The opengl.org seems to be down at the moment, but glNormalMatrix is the top left 3x3 portion of the modelview matrix transposed correct? Due to perspective projection performed to the vertices, its posible to see the side of a polygon which would normally be facing away from you if viewed in non-perspective projection. As the normal is only multiplied by forms of the modelview (as apposed to modelviewprojection) negative z maybe possible (i could be very wrong here...)

If that is not correct i must be doing something wrong, since when i render to the screen negative normal.z values only, parts of the screen is rendered in shaders of grey (albeit only very few).

Hogwash: After a 2nd or 3rd read of your message, i may have misinterpreted you... hmmm

Yeah, because of normal mapping you can get normals with negative z in screen-space. Instead of storing x and y in your texture, have you tried storing x^2 and y^2? Because they have to be positive you could pre-multiply your sign value in one of those channels. Then the shader is something like this:

float2 t = tex2D(texture, uv);float x2 = abs(t.x);float z = sqrt(1 - x2 - t.y) * (t.x / x2);

I've not used shaders in a while - there might be a sign function to get rid of the division for you.

As for the normal.z being negative in screen space, look at the following diagram:

 _/  \     X (camera)

The slashes are two faces that share a normal (the underscore). From the camera's view (X) the normal would be pointing away even if the lower face would be visible.

The problem with storing x² and y² is that you'd lose the sign, so you could still only store the sign of two coords.

Edit: An orientation (direction) can be expressed by 3 euler angles: roll, yaw and pitch. For a normal you don't need roll (since it's only the rotation around its axis). So you could store the normal as yaw and pitch and transform a uniform vector (e.g. (0,0,1)) with those angles. Then you have your normal that needs to be transformed to whichever space you need.

Apologies for the short circuit ;)

You could express the normals in object space with angles that you use to transform a base vector like (0,0,1) to get the normal, e.g. the normal (0.7071, 0.7071, 0) could be expressed by the rotation 45° pitch and 90° yaw, i.e. you first rotate the vector by 45 degrees around the x-axis and then by 90 degrees around the y-axis.

So you calculate those two angles and write them to the texture. In your deferred shader you rotate the base vector (0,0,1) accordingly to get the normal (0.7071, 0.7071, 0) in object space. You then have to transform it to whichever space you need to (world space, eye space, screen space).

Edit: The correct term just popped back into my mind: polar coordinates. Have a look at this wikipedia article.

Pola Coordinates aye.. i like the sounds of that, assuming they wont be expensive to pack/unpack. I'll look into them for sure!

But i still find myself asking the question, how have others done it with their deferred shader engines when deriving the z element. Two references suggest it:

Deferred Rendering in Killzone and 6800_leages_deferred_shading

I also wonder how many have implimented this without realizing the error. It's especially notable toward the edges of the window.

Cheers guys!

I'm not quite sure what you are trying to do. However, if you just want a higher precision representation, why not just use a higher precision internatl format?

Even in the old days, we already have 16 bit integer avaliable. And, since Geforce 6 series, we have hardware suppport 16 bit floating point value. There is simply no reason for you to build custom code to extend 8 bit value to higher precision representation, the hardware vendor had done it already.

Quote:

I'm not quite sure what you are trying to do. However, if you just want a higher precision representation, why not just use a higher precision internatl format? Even in the old days, we already have 16 bit integer avaliable. And, since Geforce 6 series, we have hardware suppport 16 bit floating point value. There is simply no reason for you to build custom code to extend 8 bit value to higher precision representation, the hardware vendor had done it already.

Well, deferred shading needs all render targets to have the same bit depth, i.e. if you use a 4x16-bit render target instead of a 4x8-bit render target, all the others would have to be 64-bit as well. This would double the memory requirements.

So since a normal has 3 components it would be perfectly fine to store only 2 of them with 16-bits in a 32-bit render target, if you can calculate the 3rd from those 2 when needed.


As far as I can see from the 2 papers you posted, hibread, the normals are in stored as screen space normals and z is assumed to be always positive. In general this is reasonable since the normal pointing away from the camera would mean that the surface can't be seen at that point (pixel) and thus should not be rendered.

However, there are some special cases like the one I posted above. The lower face would be fully rendered though the surface at the edge (where the normal is) would not be visible in reality (since a shared normal would mean a curved surface).

What about another approach? What if you just add 3 to one of your coordinates to tell the shader of z's sign? Since a normal should always have the length 1, all coordinates must be in the range -1 to +1. If a coordinate is outside that range, you have the information on the sign of z and bring the coord back into that range.

Let's say you encode the sign into the x-coord (adding 3 means z is negative):

if(x > 1) { x = x - 3; z = -sqrt(1 - x*x - y*y);}else{ //x = x; z = sqrt(1 - x*x - y*y);}Some examples (X is the adjusted value, x is the real x-coordinates) for the "special" values -1, 0, 1. The same holds true for all the fractions in between.z is positive:X = -1 -> x = -1X =  0 -> x =  0X =  1 -> x =  1z is negative:X =  2 -> x = -1 // 2 - 3 = -1 X =  3 -> x =  0 // 3 - 3 =  0X =  4 -> x =  1 // 4 - 3 =  1

This is easy to encode/decode and should leave you with enough precision for storing your coordinates.

Quote:

Original post by ma_hty I'm not quite sure what you are trying to do. However, if you just want a higher precision representation, why not just use a higher precision internatl format? Even in the old days, we already have 16 bit integer avaliable. And, since Geforce 6 series, we have hardware suppport 16 bit floating point value. There is simply no reason for you to build custom code to extend 8 bit value to higher precision representation, the hardware vendor had done it already.

Using multiple render targets in opengl limits you in what you can do. Each target needs to have the same number of bits and the same components. This, direct from the framebuffer_object extension:

"All images attached to the attachment points COLOR_ATTACHMENT0_EXT through COLOR_ATTACHMENTn_EXT must have the same internal format."

I believe with DX you can mix and match different numbers of components as long as the number of bits per target remains the same.

So what ive decided to do is pack two 16-bit fixed point values (range 0.0 to 1.0) into four 8-bit fixed point components to get more precision; which works wonders for the quality! And then later deriving the Z value. I could obviously store the Z value within another target, but im trying to limit memory usage and bandwidth as much as possible, while only incurring small amounts of extra shader processing. Actually, the extra processing isn't too bad when you consider the following: When storing all 3 components (xyz) you end up needing to re-normalize the result again during the lighting pass due to small precision errors. When using the "deriving z" approach (z = sqrt(1 - x^2 - y^2)), the length of the resultant vector is already normalized because of that equation, hence you do not need to re-normalize.

Quote:

Original post by Lord_Evil As far as I can see from the 2 papers you posted, hibread, the normals are in stored as screen space normals and z is assumed to be always positive. In general this is reasonable since the normal pointing away from the camera would mean that the surface can't be seen at that point (pixel) and thus should not be rendered. However, there are some special cases like the one I posted above. The lower face would be fully rendered though the surface at the edge (where the normal is) would not be visible in reality (since a shared normal would mean a curved surface).

Lets first define what screen space actually means. I am assuming you mean just storing the interpolated varying variable from the vertex shader constructed using gl_NormalMatrix * gl_Normal? But this sounds more like "eye" space coordinates, so i very well maybe storing incorrectly transformed normals for my intended purpose.

Concerning your quote above im getting mixed feelings on what the sign of z should be. You're right in saying that if the normals z value is negative, then that particular portion of the surface is facing away from you and should not be in view. But the reality is with normal mapping and the particular way im construction the stored normals (a varying variable gl_NormalMatrix * gl_Normal) there will be cases (quite a few toward the edges of the screen) where z is negative. This maybe just how it is, or I may be incorrectly transforming the normal into the "wrong space". Even if there was another transformation method which guaranteed that all visible triangles had normal.z values always positive (the triangle itself, not taking into account interpolated per-vertex normal), you can never make a guarantee that interpolated normals per-vertex mixed with normal mapping will follow that same theory.

Your idea with "adding 3" does sound quite good, but i have a couple of questions regarding it. With a standard RGBA internal format using type UNSIGNED_INT_8_8_8_8, you are limited to only being able to store values between 0.0 and 1.0 in increments of 256 (2^8). I did initially try to play around with using one bit as the sign (stealing 1-bit from the specular power component, reducing the remaining precision to 2^7 (128 discret values) which should be enough to describe shininess) but this got a bit ugly (not to say that it wouldn't work).

The second question is regarding the if statement. If i remember correctly, branching is a bit limited still and all branches are effectively computed, discarding the un-needed one (unless all batched fragments end with the same branch path. I forget how that works...). If this is the case, i dont like the idea of the extra computation overhead.

I like your idea about polar coordinates. Polar coordinates normally use 3 variables to describe location, but since we are dealing with a unit sphere, one of those variables is redundant. Assuming the computational expense isn't too great, it should be a goer! After i've finished writing this, ill work on it.

Cheers!

Edit: Actually... with regards to the "adding 3". Because of the limitation of values being stored having the range 0-1, you'd merely have to just scale the value back to between 0-1, then while un-packing un-scale it. By doing that, you are reducing the precision by quite a bit, but it would be do-able.

[Edited by - hibread on November 14, 2007 8:35:01 AM]

AFAIK you can store higher values than 1 in a 2x16-bit FP render target.

Regarding your concerns about the if statement: converting polar coordinates to cartesian coordinates would involve some sin/cos calculations which might it more costly than executing both branches, which would mean 1 or 2 more instructions.

Quote:

Original post by Lord_Evil AFAIK you can store higher values than 1 in a 2x16-bit FP render target. Regarding your concerns about the if statement: converting polar coordinates to cartesian coordinates would involve some sin/cos calculations which might it more costly than executing both branches, which would mean 1 or 2 more instructions.

The thing is I'm not literally using 2x16-bit floats as opengl's framebuffer extension does not allow you to mix internal format types with a particular FBO. What i am doing is packing 2x 16bit floats (with range 0.0-1.0) into 4x 8bits.

Anyway, I've implemented the spherical polar coordinate with some luck! The graphical result is perfect; no issue there what so ever. The performance is also pretty reasonable. 3x 8bit variation with the re-normalization is only slightly faster (no numerical results, just measuring using the eye) than 2x 16bit packed/unpacked as polar coordinates.

Cheers for your assistance Lord_Evil, one hell of a good idea! :)

I'm glad I could be of service [grin]

Look, I'm just trying to be nice. Please forgive me if I'm being mean.

Whenever you did something strange for performance, remember to verify whether it actually help or not. Did you do your job with the easier way first? Isn't that really necessary for you to claim extra overhead to save memory space? The cost of custom code is far beyond the code itself. You should have a reason far beyond 'just' to justify your action. (At least... I'm not interested in saving memory space at all. The hardware vendor double the video memory of GPU every year. )

By the way, concerning GPU manipulation, there is nothing you can do with Direct3D but not OpenGL. Both Direct3D and OpenGL are just API that wrapping the GPU functionalities. In this sense, they are just the same (if you didn't make obvious mistakes). And, actually, most new technology available on GPU are supported via OpenGL extension first and then port to Direct3D later on.

Anyway, if you insist, keep on.

Wish you good luck.

You're correct that the normal z component can be negative after a normal map has been applied. This normally isn't a big problem because the normal map only perturbs a normal by a small amount, so you're only ever going to have a small negative z component (ie. close to zero) in this case. Give it a shot and see what the results look like, because this is exactly the way that Killzone (Gueruilla) is doing it with their deferred renderer.

I think a few people here are confusing screen space with view space. View space is a space in world units, whereas screen space is in pixel units (ie. z always is zero).

Your idea about using polar coordinates is interesting, but there isn't really an efficient way of converting to/from Cartesian. Here is my implementation-

float3 ConvertPolarToCartesian(float2 polar){	float2 sinPolar;	float2 cosPolar;	sincos(polar, sinPolar, cosPolar);	return float3(		cosPolar.x * cosPolar.y,		sinPolar.x * cosPolar.y,		-sinPolar.y	);}float2 ConvertCartesianToPolar(float3 cartesian){	static const float twoPi = 2.0f * 3.14159f;	return float2(		atan2(cartesian.x, cartesian.y),		twoPi - atan2(length(cartesian.xy), cartesian.z)	);}

[Edited by - hogwash on November 14, 2007 11:27:25 PM]

Hey hogwash, thanks for your replies!

Here is an illustration I just created to demostrate where and how normal.z can be negative (using some insane artistic talent, I must admit).

The blue vector is merely the view direction to the surface in question. The green arrow (a normal) is the basic surface normal which has quite a negative z value (in modelview space, not modelviewprojection). The purple normal is an example of a modified normal because of a normal map and it has quite an aggressive negative z value.

The grey dotted line denotes the reflection plane when using the standard z = +sqrt( 1 - x^2 - y^2) to produce the calculated normal (brown arrow) that would be used for any further lighting calcualtions. This would produce incorrect lighting for that entire face.

I have a feeling im using the wrong coordinate space to store the normals as this problem seems quite significant. Significant enough for professional title programmers not to concern themselves with it. Either that, or I'm not visualizing the different spaces correctly!



Concerning trig functions, does anyone have a clue on how expensive they "should" be? Are they efficient in the sence they use look up tables maybe?

After thought: The field of view in the example is probably about 100-110 degrees. If you go to the extreme with 180 degrees, its possible for the normal.z value to be near on -1. If you then add a very drastic normal map to that (add another ~90 degrees), normal.z is back to ~0 again!