Light Propagation Volumes and SH questions

Quote:

Original post by jcabeleira- If each color component requires 4 SH coeficients, how can I encode this into a single RGBA volume texture? Perhaps I'll need a different volume texture for each color component?

You need one volume texture per color channel.

Quote:

Original post by KRIGSSVIN Quote: Original post by jcabeleira- If each color component requires 4 SH coeficients, how can I encode this into a single RGBA volume texture? Perhaps I'll need a different volume texture for each color component? You need one volume texture per color channel.

Thanks!

About the SH coeficients, isn't true that the 4 coeficients only allow the representation of a single light direction?

No, with 1-band spherical harmonics you can express (inaccurately) a combination of signals along x, y, z.

Also, take care when writing SH into the RGBA volume texture, because the values can be negative and also outside of [-1, 1] when you're blending.

Quote:

Original post by oligopod No, with 1-band spherical harmonics you can express (inaccurately) a combination of signals along x, y, z.

Yeah, but it looks like crap. For more or less adequate SH lighting, you'll need 9 coefficients per colour channel.

Quote:

Original post by Yann L Yeah, but it looks like crap. For more or less adequate SH lighting, you'll need 9 coefficients per colour channel.

Indeed, but storing 9 coeficients in a texture or group of textures is a pain in the ass. To avoid this, I was thinking about dropping the SH and using a combination of axis, similar to the one Valve used for their normal mapped radiosity algorithm, to define the space of incoming light. I think that with a good choice of 4 axis, it's possible to represent a wider range of light directions than SH using only 4 coeficients (one for each axis) for each color component.

Hello,

I'm also looking into the Light Propagation Volume technique. Like jcabeleira i have also got a similar technique working but only with colour.

I have a few questions which i hope someone can help me with; i will be referring from Crytek's Siggraph 09 course, which can be found here.:

1) When working out the weights for the surfel, the paper suggests that Ws = Ssplat/Scells = tcells/t. I thought tcells and t were the same and what is Ssplat?

2) How to convert the volume data into values that i can just add into the final scene i.e from SH to the light and colour value for the fragment that i'm wanting to shade.

Thanks

EDIT: Doh! Okay, just found out that tsplat is explained in the paper.

[Edited by - manxankur on January 23, 2010 3:31:53 PM]

Quote:

Original post by manxankur 2) How to convert the volume data into values that i can just add into the final scene i.e from SH to the light and colour value for the fragment that i'm wanting to shade.

This is done in a deferred pass. For each pixel, you use its world-space position to sample the propagation volume (you must convert the position into the corresponding texture coordinate of the volume), this gives you a Spherical Harmonic function. Then you use the pixel's world-space normal on the SH function to get the pixel's illumination value.

Sorry if my explanation is not very detailed. I only know the theory but I haven't yet implemented this technique with SH.

Quote:

Original post by jcabeleira Then you use the pixel's world-space normal on the SH function to get the pixel's illumination value.

Hi jcabeleira,

When you say "use the pixel's world-space normal on the SH function" do you mean dot product the normal with each of the SH functions (from the different colour channels RGB)?

Also, what is going on in the propagation shader that Crytek have included in the appendix in their paper? I've tried to understand it but i'm new to SH. From what i can tell, they're:
- getting the adjacent SH value,
- then they calculate the incoming direction SH,
- dot product the adjacent SH and the incoming direction SH to get a incident luminance,
- Add the incident luminance value to the current SH.

Is the dot product of a SH function the same as a vector? What does the scale do to the luminance and incoming direction SH in the last part?

Quote:

Original post by manxankur When you say "use the pixel's world-space normal on the SH function" do you mean dot product the normal with each of the SH functions (from the different colour channels RGB)?

I understand why are you confused, this SH stuff is not intuitive. I took a lot of time to grasp the ideia behind this technique, and after all this time there are still some things I dont understand very well. The main problem is that Crytek's document is not very well written and is lacking images to explain the mathematical details.

Anyway, you don't use a dot product, but something similar. When you sample the voume texture, you get a texel with 4 components and each component represents an SH coeficient (c0, c1, c2 and c3). If I'm not mistaken, you combine the normal and SH this way:

float result= c0 + c1*normal.y + c2*normal.z + c3*normal.x;

If you're having troubles understanding what the SH coefficients mean, think of them this way (although it isn't mathematically correct, but who cares?):
The c3, c1 and c2 coefficients represent how much light goes along each of the x, y and z axis directions respectively. And the c0 represents how much light comes from all directions.

Quote:

Original post by manxankur Also, what is going on in the propagation shader that Crytek have included in the appendix in their paper? I've tried to understand it but i'm new to SH. From what i can tell, they're: - getting the adjacent SH value, - then they calculate the incoming direction SH, - dot product the adjacent SH and the incoming direction SH to get a incident luminance, - Add the incident luminance value to the current SH. Is the dot product of a SH function the same as a vector? What does the scale do to the luminance and incoming direction SH in the last part?

That's the tricky part that they explain very poorly. I would need to make a sketch to explain it to you properly but I'll try do describe it in text.

First, you should remember that when lights are injected into the volume they occupy one single texel, and the purpose of the propagation steps is to expand each light until it forms a sphere of light that represents a point light.

The propagation step should be intuitive, for each texel you get the neighbour texels and combine them. The only thing hard to understand is the combination method.
The strange combining method is used to solve the following problem: after one texel receives the SH function from one of its neighbours, how do you avoid that
that function propagates back to the original texel on the next propagation step? If every texel propagates its SH function in all directions, each texel will end up receiving, from its neighbours, the function that is has previously propagated.
To avoid this problem, each propagated SH function is "cutted" to represent only light going through the neighbour texel direction. For example, if an SH is being propagated into the left neighbour texel, it will be cut to represent light going only into the left direction. This way, on the next propagation step, the SH can't be propagated back to it's original texel on the right direction because it only represents light going into the left direction.

The way the light is cut uses a mathematical property of SH. This property says that if you have two SH functions, one that represents your lighting and another that represents a cone pointed at the neighbour texel, and if you apply the Spherical Harmonics dot product on the two SH functions, you get an SH function that represents only the lighting from your original lighting function that goes into the neighbour texel's direction.

I hope my explanation was clear. :P

[Edited by - jcabeleira on January 24, 2010 3:19:34 PM]

Hi,

Thanks for the explanation jcabeleira. I tried this:

Quote:

Original post by jcabeleira float result= c0 + c1*normal.y + c2*normal.z + c3*normal.x;

but it didn't work so well. Instead i'm now doing a dot product between the normal of the current fragment (from the camera's view perspective) with the SH sample from the volume. This is giving me a better results.

So, i've nearly got this working but i'm still having one little issue. Here's the screenshot:



This is from a birds eye view of a grid with two coloured sphere on either end of the grid. The two smaller versions of the grid and spheres are from the light's perspective. You should also be able to see two dark lines on the main grid; these lines are perpendicular to the light's view direction. Does anyone have any idea of what these two dark lines are?

Thanks

Quote:

Original post by manxankur Hi, Thanks for the explanation jcabeleira. I tried this: Quote: Original post by jcabeleira float result= c0 + c1*normal.y + c2*normal.z + c3*normal.x; but it didn't work so well. Instead i'm now doing a dot product between the normal of the current fragment (from the camera's view perspective) with the SH sample from the volume. This is giving me a better results. So, i've nearly got this working but i'm still having one little issue. Here's the screenshot: This is from a birds eye view of a grid with two coloured sphere on either end of the grid. The two smaller versions of the grid and spheres are from the light's perspective. You should also be able to see two dark lines on the main grid; these lines are perpendicular to the light's view direction. Does anyone have any idea of what these two dark lines are? Thanks

I have no ideia, but I think you should learn about SH in more detail before trying to implement this technique. If you don't do it, you'll always face problems that you can't solve because you don't understand them.

I know a few articles about SH for videogame programming that you could read. If you have got some mathematical background, you should understand the contents of these articles and thus understand your own implementation better.
http://www.cs.umd.edu/class/spring2005/cmsc828v/papers/spherical-harmonic-lighting.pdf
http://www1.cs.columbia.edu/~ravir/papers/envmap/envmap.pdf

Rotate the SH representation of a cosine lobe in the direction of ( normal in worldspace * -1.0 ). Since the cosine lobe is a zonal harmonic, this has a simple analytic formula. Then do a dot product between this and the SH sample from the volume, this will give you an irradiance value.

Here's where you can find how to rotate the zonal harmonic:

http://www.ppsloan.org/publications/StupidSH35.pdf

Good luck with your implementation :)

Quote:

Rotate the SH representation of a cosine lobe in the direction of ( normal in worldspace * -1.0 ). Since the cosine lobe is a zonal harmonic, this has a simple analytic formula. Then do a dot product between this and the SH sample from the volume, this will give you an irradiance value.

This is what i am doing now.

Quote:

Here's where you can find how to rotate the zonal harmonic: http://www.ppsloan.org/publications/StupidSH35.pdf

That's a good read, it's helped me a little more in understanding SH, although i still need to do more reading to get comfortable with it.

I've finally got it to work, i had to tweak some of the values and now it looks okay.

Quote:

Original post by manxankur I've finally got it to work, i had to tweak some of the values and now it looks okay.

Nice, could you show some screenies?

Quote:

Original post by Yann L Quote: Original post by oligopod No, with 1-band spherical harmonics you can express (inaccurately) a combination of signals along x, y, z. Yeah, but it looks like crap. For more or less adequate SH lighting, you'll need 9 coefficients per colour channel.

SH lighting is a pretty broad category. You can get good lighting with 4 coefficients it just depends on what you are talking about.

If you are talking about convex Lambertian objects under distant incident lighting then you can conserve up to 87.5% or the signal using an order 1 four term SH basis.

Order 2, using 9 coefficients, conserves 98% of that signal. Odd terms of l > 1 vanish (go to 0). The difference is minimal and perceptually hard to tell the difference.

Now, if you are trying to capture higher frequency signals such as soft shadows then order 1 will not represent the signal as well as order 2. If you are trying to represent glossy objects then order 2 will not represent the signal and order 4 is necessary. Sharp shadows take somewhere above a 1000 coefficients.

Though, my point is that order 1, 4 coefficients, can represent very well simple diffuse Lambertian lighting. It really depends on what you are trying to model.

-= Dave

Quote:

Original post by David Neubelt SH lighting is a pretty broad category. You can get good lighting with 4 coefficients it just depends on what you are talking about. If you are talking about convex Lambertian objects under distant incident lighting then you can conserve up to 87.5% or the signal using an order 1 four term SH basis. Order 2, using 9 coefficients, conserves 98% of that signal. Odd terms of l > 1 vanish (go to 0). The difference is minimal and perceptually hard to tell the difference. Now, if you are trying to capture higher frequency signals such as soft shadows then order 1 will not represent the signal as well as order 2. If you are trying to represent glossy objects then order 2 will not represent the signal and order 4 is necessary. Sharp shadows take somewhere above a 1000 coefficients. Though, my point is that order 1, 4 coefficients, can represent very well simple diffuse Lambertian lighting. It really depends on what you are trying to model. -= Dave

That is all true. The problem is that a single texel of a light propagation volume may have to represent light comming from several directions, and you can't encode that using only 4 coeficients.
If you try to encode complex illumination like that using only 4 coefficients you're even likely to obtain bad light propagation during the propagation steps.

Which is why you have to do the "smart downsampling" of the reflective shadow maps - this ensures that you get surfels with only a few orientations being injected into the volume. It does work pretty well if you try it.

Quote:

Original post by oligopod Which is why you have to do the "smart downsampling" of the reflective shadow maps - this ensures that you get surfels with only a few orientations being injected into the volume. It does work pretty well if you try it.

´

I admit I haven't implemented this, but I think the downsampling filter has nothing to do with reducing the amount of orientations injected on the volume. The downsampling filter simply takes a high resolution picture of the scene and downsamples it while preserving most of the information for better processing and to increase the effect's visual stability.

It is also true that when lights are injected into the volume, the SH only have to represent simple orientations. It is during the propagation stage that texels start encoding light coming from various directions and that can't be avoided in any way. I think that the only reason the 4 coeficients are usually enough is because Crytek uses small amounts of propagation steps (6 if I'm not mistaken), but if you use more steps you'll start seeing lights overlapping each other which will force the SH functions to encode more light directions that they can take.

By the way, if any of you guys has already implemented this technique, could you post a few images demonstrating it? I'd like to know if this technique really provides good global-ilumination effects, because the videos from Crytek didn't convinced me. I've also read made some experiments with this technique, although I didn't use SH, and I wasn't convinced either.

I have implemented it, but unfortunately I can't post details related to my particular version due to NDA.

The downsampling does give you fewer light directions, essentially you take a 2x2 block of the RSM, select the brightest surfel and then *average* all the surfels that fall in the same cell. That averaging includes the normal too.

Anyway, out of all the shortcuts and approximations that this technique has - e.g. the extremely sparse RSM and volume texture - I'd say the SH representation is the least objectionable.