Original Post
I've been messing up some time with rendering a mesh with multiple see-through textures, until I dived into the theory. This is what I came with :
1) rendering multiple layers
We can define the visible colour Vi of layer i, as the colour that would be seen if layers 0 to i were visible, and the remaining layers were invisible. Vi is obtained by computing, from bottom (layer 0) to top (layer i), the successive values of Vi with the formula :
Vi = Ci. αi + Vi-1.(1 - αi)
with Ci the colour stored in layer i, and αi the alpha value stored in layer i.
This is the classical blending formula, but the destination colour is not the colour of the underlying layer, it is its visible colour. The lowest layer is the background, its alpha is by definition 1, so V0 = C0.
This is easy to implement in a pixel shader: we need just to compute the successive values of Vi, from layer 0 upwards, possibly skipping a hidden layer. This is well known stuff.
2) merging layers
Sometimes we want to merge two layers together, or better said replace two textures with their blended colours. This is the case, for example, when we paint with a textured brush on one layer. At first sight, all we have to do is use the classical blending formula (αdest, 1-αdest). Doing this we get a "halo" around the brush (where the alphas are somewhere between 0 and 1). The reason is that we ignore the source alpha, and we shouln't.
Let us see how to correctly merge two successive layers together. What we want is to replace the two layers with one, and get the same visual result than when they are layered. With the two layers, say layers 3 and 4, the visible colour as defined above is given by:
V3 = C3. α3 + V2.(1-α3)
V4 = C4. α4 + V3.(1-α4)
and we want to replace this by layer 3’ (merge of layers 3 and 4) :
V’3 = C’3. α’3 + V2.(1-α’3)
so that V’3 = V4
Therefore we can write the identity :
C’3. α’3 + V2.(1-α’3) = C4. α4 + V3.(1-α4)
= C4. α4 + (C3. α3 + V2.(1-α3)).(1-α4)
= C4. α4 + C3. α3. (1-α4) + V2.(1-α3).(1-α4)
Since V2 is an invariant, this can only be met if two conditions are satisfied :
C’3. α’3 = C4. α4 + C3. α3. (1-α4)
1-α’3 = (1-α3).(1-α4)
α is the inverse of the opacity : the transparency, so the merged layer’s transparency is the product of the two layer’s transparencies, or :
α’3 = 1- (1-α3).(1-α4)
And the merged color :
C’3 = (C4. α4 + C3. α3. (1-α4)) / α’3
So far so good. I'd welcome comments on this by more experienced people.
My problem is to implement this formula in glsl. I cannot use the existing openGL blending formulas, so I need to do it with GLSL. The problem is that I need to write the result of the merge in texture 3 corresponding to layer 3, but I also need a lookup in texture 3. Can I do simultaneous read and write on the same texture ? I guess not. Another solution would be to write in a temporary texture, and replace texture 3 by this texture when the blending is done. Considering my brush application, I need this to be quick (of course). Is there another option ? What's the best way to do this ? Any help/advice welcome.
1) rendering multiple layers
We can define the visible colour Vi of layer i, as the colour that would be seen if layers 0 to i were visible, and the remaining layers were invisible. Vi is obtained by computing, from bottom (layer 0) to top (layer i), the successive values of Vi with the formula :
Vi = Ci. αi + Vi-1.(1 - αi)
with Ci the colour stored in layer i, and αi the alpha value stored in layer i.
This is the classical blending formula, but the destination colour is not the colour of the underlying layer, it is its visible colour. The lowest layer is the background, its alpha is by definition 1, so V0 = C0.
This is easy to implement in a pixel shader: we need just to compute the successive values of Vi, from layer 0 upwards, possibly skipping a hidden layer. This is well known stuff.
2) merging layers
Sometimes we want to merge two layers together, or better said replace two textures with their blended colours. This is the case, for example, when we paint with a textured brush on one layer. At first sight, all we have to do is use the classical blending formula (αdest, 1-αdest). Doing this we get a "halo" around the brush (where the alphas are somewhere between 0 and 1). The reason is that we ignore the source alpha, and we shouln't.
Let us see how to correctly merge two successive layers together. What we want is to replace the two layers with one, and get the same visual result than when they are layered. With the two layers, say layers 3 and 4, the visible colour as defined above is given by:
V3 = C3. α3 + V2.(1-α3)
V4 = C4. α4 + V3.(1-α4)
and we want to replace this by layer 3’ (merge of layers 3 and 4) :
V’3 = C’3. α’3 + V2.(1-α’3)
so that V’3 = V4
Therefore we can write the identity :
C’3. α’3 + V2.(1-α’3) = C4. α4 + V3.(1-α4)
= C4. α4 + (C3. α3 + V2.(1-α3)).(1-α4)
= C4. α4 + C3. α3. (1-α4) + V2.(1-α3).(1-α4)
Since V2 is an invariant, this can only be met if two conditions are satisfied :
C’3. α’3 = C4. α4 + C3. α3. (1-α4)
1-α’3 = (1-α3).(1-α4)
α is the inverse of the opacity : the transparency, so the merged layer’s transparency is the product of the two layer’s transparencies, or :
α’3 = 1- (1-α3).(1-α4)
And the merged color :
C’3 = (C4. α4 + C3. α3. (1-α4)) / α’3
So far so good. I'd welcome comments on this by more experienced people.
My problem is to implement this formula in glsl. I cannot use the existing openGL blending formulas, so I need to do it with GLSL. The problem is that I need to write the result of the merge in texture 3 corresponding to layer 3, but I also need a lookup in texture 3. Can I do simultaneous read and write on the same texture ? I guess not. Another solution would be to write in a temporary texture, and replace texture 3 by this texture when the blending is done. Considering my brush application, I need this to be quick (of course). Is there another option ? What's the best way to do this ? Any help/advice welcome.
