Skip to main content
GameDev.net gamedev.net
🔒 Locked

Drawing alpha-tested foliage nicely (WARNING: image heavy)

Started by PlayerX Jul 14, 2007 at 10:05 AM 37 replies 24.9k views
Original Post
PlayerX
PlayerX
Hey there. I came across this little trick a couple of weeks back which has been a god-send for me when rendering alpha-tested foliage. Basically it allows alpha-tested foliage to look as good as alpha-blended foliage without the sorting artifacts. I still see a lot of commercial and not-so-commercial games using straight ugly alpha-tested foliage that I figure I'm not the only one this could help. Anyway, ideally we want to draw our trees using alpha-blending so we get nice smooth edges. However, we need to draw the surfaces from farthest to closest or we get artifacts: So usually we use alpha-testing as we can't really afford the time to sort all those thousands of surfaces we're using to draw our trees with. Here's the same scene using alpha-testing with a cut-off of 0.5: As you can see the problem with alpha-tested foliage is the hard edges. The foliage tends to shimmer when moving as well. But at least it's simple and fast. This is what most games seem to use. We can combine both alpha-blending and alpha-testing in the same pass but then we get fringes which can be unsightly: Now here's the scene rendered using the trick I came across: As you can see, we get all the benefits of alpha-blended foliage with the simplicity of alpha-tested foliage, and sure looks nicer than either. Here's the trick (using OpenGL lingo). First render all your foliage using standard alpha-testing, but with a fairly high alpha cut-off:
    glBlendFunc( GL_ONE, GL_ZERO );
    glEnable( GL_ALPHA_TEST );
    glAlphaFunc( GL_GREATER, 0.75 );
    ...draw foliage...
Now redraw your foliage using alpha-blending but with a low alpha-test cut-off, depth-writes turned off, and depth testing set to less-than:
      glAlphaFunc( GL_GREATER, 8.0 / 255.0 );
      glBlendFunc( GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA );
      glDepthMask( GL_FALSE );
      glDepthFunc( GL_LESS );
      ...draw foliage again...
      glDisable( GL_ALPHA_TEST );
      glDepthMask( GL_TRUE );
      glDepthFunc( GL_LEQUAL );
As you can see there is the cost of drawing all your foliage twice, but I think the image quality makes it worth it. You don't need to disable depth writing after each tree - draw all your organic foliage using the alpha-testing pass, then do it all again with the alpha-blending pass. Here's another comparison shot using a pine-tree (alpha-blended, alpha-tested, both): Static screenshots don't really do it justice. It looks nicer in motion. Anyway, hope that helps someone as it sure helped me. Apologies if this is a known trick. If that's the case, I figure there are some who don't know it.
jamesw
jamesw
Nice pictures! Here's a link to a similar technique. Unfortunately the extra pass is always the first to go when I get around to optimizing because it uses up so much fill rate.
Jack Sotac
Jack Sotac
Does anyone know if this method can be combined with the ones above? It's for billboards but might apply here.
0xa0000000
Promit
Promit
I saw a cool trick in GPU Gems 2 that dealt with the same problem, but I was just skimming in a bookstore and I don't remember what it was. Anyone have the book handy? It was right near the beginning -- I believe it was the grass chapter.
SlimDX | Ventspace Blog | Twitter | Diverse teams make better games. I am currently hiring capable C++ engine developers in Baltimore, MD.
AndyTX
AndyTX
The most robust solution I've seen is to use alpha-to-coverage which is a simple flag enable. It isn't perfect (it's effectively sub-pixel "screen-door" anti-aliasing) and needs to be used with MSAA, but it gives you N levels of free sorting per-pixel when N is the number of MSAA samples.

I've seen the technique in the OP before actually and it's usable, however it has several artifacts that you can actually see in the post. In particular note the left side of the top middle branch in the left-most image on the bottom row of images. (... Wow! Hopefully you can decode that.)

Still it may be preferable to typical alpha testing which looks even worse in many situations.

[Edit] The trick they use in Gems 2 is also a "screen-door" effect, but at the pixel level rather than sub-pixel so it won't look as good as alpha-to-coverage, and is more expensive to boot.
Kern_d
Kern_d
I think the artifact you refer to are actually present on other images and using other technique, they seem to be texture wrapping artifact.

That is if you talk about the pixels in the air?
Rompa
Rompa
Nice. In your second pass, would it not make a little more sense to render with alpha LEQUAL 0.75?
BTW, I think Tom Forsyth described this technique about 5 years ago on the DX or GA mailing list. As with anything computer graphics related, its pretty hard to invent something that hasn't been done before :-(
PlayerX
PlayerX
So it was a known technique. Wish I'd found it before now. I definitely get the impression it's not a widely known trick because there are a lot of games that have ugly aliased alpha-tested foliage. But then, as jamesw says, that could simply be a case of not wanting to have to draw it all twice.
Quote:
I've seen the technique in the OP before actually and it's usable, however it has several artifacts that you can actually see in the post. In particular note the left side of the top middle branch in the left-most image on the bottom row of images. (... Wow! Hopefully you can decode that.)

As Kern_d mentioned, it looks like a wrapping artifact. The models I used aren't the best.
Quote:
Nice. In your second pass, would it not make a little more sense to render with alpha LEQUAL 0.75?

Give it a go. I haven't analysed the technique much, just enough to verify it works. I didn't invent the trick, just encountered it being used by an old game and thought I'd share it as it helped me a lot.
Quote:
The most robust solution I've seen is to use alpha-to-coverage which is a simple flag enable.

I did play with alpha-to-coverage a while ago but wasn't very impressed. I can't recall if I used antialiasing with it or not. For me personally I found the trick I described here produced nicer results. I might go revisit a2c with antialiasing now though and see if it makes a difference.
Matt Aufderheide
Matt Aufderheide
alpha-to-coverage doesnt even work at all without anti-aliasing--it is only activated if AA is enabled.
AndyTX
AndyTX
Quote:
Original post by PlayerX
As Kern_d mentioned, it looks like a wrapping artifact. The models I used aren't the best.

Fair enough, although I'm pretty sure artifacts could occur depending on the "threshold" values that you choose. At the very least banding near those thresholds. Now of course whether a degenerate case can be constructed (I'm almost positive that it can) and whether it comes up in practice are two different things :)

Quote:
Original post by PlayerX
I did play with alpha-to-coverage a while ago but wasn't very impressed. I can't recall if I used antialiasing with it or not. For me personally I found the trick I described here produced nicer results. I might go revisit a2c with antialiasing now though and see if it makes a difference.

Yeah A2C will do absolutely nothing without MSAA enabled since it relies on writing the alpha value to the MSAA sample "coverage", which in turn uses the MSAA hardware to store and sort the N samples, averaging them at the end (if you use a standard resolve... in D3D10 you could actually do whatever you want with the samples after the fact).
PlayerX
PlayerX
Alpha-to-coverage seemed to work without AA, in that it drew the transparent areas using some kind of dithering. That's why I wasn't impressed. It definitely sounds like I didn't use it in its best light though, so will have to have another go.
AndyTX
AndyTX
Quote:
Original post by PlayerX
Alpha-to-coverage seemed to work without AA, in that it drew the transparent areas using some kind of dithering. That's why I wasn't impressed. It definitely sounds like I didn't use it in its best light though, so will have to have another go.

Hmm interesting, I just assumed it would do nothing w/o the anti-aliasing hardware being active. In any case, sub-pixel screen-door should certainly still look better than otherwise, so give it a go and see if you like the results.
PlayerX
PlayerX
I had the opportunity to throw in alpha-to-coverage tests. Here are the results. Again, excuse the artifacts at the edges of the leaves; this is a texture clamping error.

Close up, respectively: alpha-blending only, alpha-testing only, blending and testing, two-pass, alpha-to-coverage.




Farther away.




Distant.




You were right, alpha-to-coverage doesn't work at all without anti-aliasing enabled - I must've had AA going when I experimented with it earlier. The above shots were taken with 16x AA on a GF7.

Personally I feel the two-pass method produces the best results. Alpha-to-coverage seemed to be barely better than alpha-testing alone as it didn't look good when moving and the stippled nature of close-ups looks nasty. It's a nice alternative, though, when you have so much foliage you can't afford to render it twice. But if you've got the odd tree or three and small clumps of scrub grass so rendering them twice would be unnoticeable, the two-pass method produces nicer results IMHO.

Do the alpha-to-coverage screen-shots reflect what you're getting, or am I doing something wrong?
AndyTX
AndyTX
Quote:
Original post by PlayerX
You were right, alpha-to-coverage doesn't work at all without anti-aliasing enabled - I must've had AA going when I experimented with it earlier. The above shots were taken with 16x AA on a GF7.

16x AA on GF7?? Try 4x for comparison since IIRC GF7 can only do 4 "real" MSAA samples (GF8 can do 8).

Quote:
Original post by PlayerX
Do the alpha-to-coverage screen-shots reflect what you're getting, or am I doing something wrong?

Those look remarkably bad compared to what I've seen actually. Unfortunately I don't have an example handy (it was at work when I was trying this all out a year or two ago), but I'll see what I can dig up when I get a chance...
zedz
zedz
in my gf7's application settings u can choose 8xAA

yes i also thought the alphato coverage shots looked worst than i though, so i fired up one of my apps that does it, and yep it does look a lot worse than i thought. though better than plain alpha testing.
the beauty with alpha to coverage though it works with depthtesting, playerX your method does also but it requires u to draw the (expensive) alpha geometry twice, thus its a lot slower
AndyTX
AndyTX
Quote:
Original post by zedz
in my gf7's application settings u can choose 8xAA

Yes but IIRC 8xS AA (or whatever it's called - something like that) is a hybrid mode that does 2x super-sampling with 4x MSAA.
PlayerX
PlayerX
Quote:
16x AA on GF7?? Try 4x for comparison since IIRC GF7 can only do 4 "real" MSAA samples (GF8 can do 8).

I tried 4x AA as well. I only showed the 16x because it looked better. GF7 can do 16x AA in an SLI configuration.
Quote:
the beauty with alpha to coverage though it works with depthtesting, playerX your method does also but it requires u to draw the (expensive) alpha geometry twice, thus its a lot slower

Yep. It's not a win when you have lots of foliage. But if you don't, then it beats the other methods in quality. What the pictures don't show above is how nice it looks when moving. Personally I reckon give the player the option of the method to use. Have a slider in the graphics settings tab for foliage; Speed vs Quality.

I've seen quite a few games that don't have much foliage though, but still use alpha-testing for them. These games could definitely benefit from the two-pass.
AndyTX
AndyTX
Quote:
Original post by PlayerX
I tried 4x AA as well. I only showed the 16x because it looked better. GF7 can do 16x AA in an SLI configuration.

Ahh, SLI, right! I always forget about that... Still looks surprisingly bad for 16 samples (that should give 16 distinct levels of alpha).

Anyways definitely always good to have another method available in the toolbox :)
Matt Aufderheide
Matt Aufderheide
Quote:
Original post by AndyTX

Those look remarkably bad compared to what I've seen actually.


That's because these shots are a worst-case scenario--black tree with fine detail against a light background.. is guarenteed to look bad with alpha-to-coverage.

In my experience, A2C is far better than this two-pass method, which is terrible for fillrate. Imagine doing a grass field with the two pass method...impossible. How often do you just have one tree standing alone?

Also this is very low-poly-looking tree; most trees in modern applications are higher poly, 5000-8000 at the nearest LOD with a LOT of overdraw for dense foliage. The two-pass method is a clear loser.

zedz
zedz
Quote:
Original post by AndyTX
Quote:
Original post by zedz
in my gf7's application settings u can choose 8xAA

Yes but IIRC 8xS AA (or whatever it's called - something like that) is a hybrid mode that does 2x super-sampling with 4x MSAA.

does it!!
strange it just saiz 8x in the settings, yould think theyld advertise it as such
after all 2x super-sampling with 4x MSAA is certainly better than 8xMSAA.

no wonder performance always dived when i turned it on

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.