[DX9] antialiasing doesn't want to work

You don't want non-maskable if you're trying to enable MSAA. D3DMULTISAMPLE_2_SAMPLES and D3DMULTISAMPLE_4_SAMPLES with quality of 0 should be supported on nearly every discrete GPU out there...what are you testing this on?

I'm sure it's not a support issue. I'm testing on an ATI HD4850.

When I use CheckDeviceSampleType (or whatever that function name was) it returns true for support all the way up to 14 samples (but not 16 oddly enough).

But like I say in the OP, I tried using masked AA with 2 samples and quality 0, but the D3D9 device creation fails saying ValidatePresentationParams failed, and the debug runtimes don't give any more info than that.

I'm slightly progressing on this issue, but still utterly confused by it.

I have managed to get it to stop failing when using D3DMULTISAMPLE_2_SAMPLES up to D3DMULTISAMPLE_8_SAMPLES, with D3DMULTISAMPLE_16_SAMPLES failing on the compatibility check, but that's expected.

However it still does no AA in the scene whatsoever.

The AA sample with the DirectX SDK work fine - AA turns on or off at the various sample levels without a problem. I've poured over the sample code, but cannot see any differences between how they implement it and how I do.

I'm at a complete loss as to what might be causing this. Right now I'd be happy to hear any wild ideas that are a million-to-one chance of being the problem, because I've exhausted all the typical problems.


EDIT: ok, I found it, and feel a bit daft for not knowing this:

Seems you can't anti-alias a render to texture surface. Or can you? If anyone knows a way to render to a texture with AA I'd love to hear it. I know you can double up the texture size, but I was hoping for something better than that.

EDIT EDIT:

Actually, never mind. I've found a way around it in my game, that doesn't involve too much extra rendering. I guess it'll do, at least until I decide to write a DX10 path :)

[Edited by - Damocles on September 20, 2009 6:09:58 AM]

Well, although you said you're done with this problem, here is one possible solution:
Render normally to the backbuffer, but don't call present. Get the backbuffer and copy it (StretchRect) to your surface.

Ah, thanks Tom, that's better than my method (I was redrawing the parts that were affected by the special fx in a second pass).

Hmm... what's the performance like on StretchRect? I'm seeing a huge performance drop using copy the backbuffer to the render-to-texture method. It's about 3 times slower than the drawing the scene a second time method. Is that normal?

You most certainly can use MSAA with a render target texture. You do it like this:

1. Create a render target surface with MSAA using IDirect3DDevice9::CreateRenderTarget
2. Create a render target texture with the same format and dimensions using IDirect3DDevice9::CreateTexture, using D3DUSAGE_RENDERTARGET
3. In your render loop, render your geometry to the surface created in step 1
4. When you're done rendering to the render target, call IDirect3DDevice9::StretchRect to resolve the MSAA render target to your other render target texture. This resolves the subsamples, so that you have normal texture that you can sample.

Ah, is that what I'm doing wrong? I'm currently rendering the MSAA content tot he backbuffer surface, then calling stretchrect to copy that surface to the normal (non-MSAA) texture surface, ready for post-render FX processing.

Should I be making a third surface, the same dimensions as the backbuffer, and draw to that instead?

I don't think it should matter much whether you StretchRect from the backbuffer or from another MSAA surface...AFAIK performance there should be the same. You could certainly try.

What exactly is the "performance drop" you're getting from using StretchRect? An MSAA resolve is by no means free, but it should be quite cheap overall. Remember that when you're determining performance costs you want to do your measurements in millseconds and not FPS, since FPS is not linear. A drop of 100 FPS is very small when you're going from 1000->900, but quite big when going from 125->25. Also remember that you usally can't get meaningful performance statistics if your app is running at a very high FPS..you have to make sure you're actually stressing the GPU a bit.

It's not so much the huge FPS drop that bothers me (although going from 500fps to 160 is pretty steep) it's the fact that it's much worse than I was getting when I was drawing the scene twice (400 fps). That just seems very wrong to me.

If I could ever figure out how to get AMD code analyst to time DX functions, I'd be able to get a more accurate account of the situation, but I haven't figured that one out yet.

However, I now think the problem might be something unrelated to stretchRect. I tried taking out the call to stretchRect and I was still getting awful framerates, but the only thing I added between the good fps and bad is the stretchRect stuff. So I'm confused as to how I've managed to destroy the performance of the app without changing anything! Just lucky I guess :)

Well, maybe a stupid advice, but I've already done this myself many times, so, just to make sure:
Didn't you for example enable the debug runtimes? It pushes the FPS down and if you forget about it, you may wonder how's that possible without changin anything in your code ;)
Also - on winxp (isn't happening on vista anymore) when I had Miranda IM with partial opacity on the screen next to the D3D window, it lowered my FPS quite a lot, also making me wonder WTH :O

160 fps is not a terrible framerate.

Rendering, as implemented in modern computers, is a series of parallel tasks. You can imagine the graphics pipeline as an actual physical pipeline, in which a "fluid" of instructions flows from CPU to the screen. Some of the pipes are smaller than others, corresponding to the varied bandwidth and processing power of the components.

If a given stage in the pipeline introduces a bottleneck (as AA usually does), you can stress the other stages for free until one of the other stages becomes the bottleneck.

Therefore, the difference between 400 fps and 160 fps is absolutely meaningless if all you do is some simple rendering. Performance tuning is only worthwhile when you actually have all your systems up and running, with system workload similar to the final target app.

When you can make your frame's "instruction flow" go through the entire pipeline at 1/60 of a second, most users will be happy.

@Tom
No, it wasn't the debugs - that's always the first thing I check when the framerate mysteriously drops (I too have been caught out by it before). And no other running apps either, I've been caught by that one as well.

@Nik
I understand all that, but I'm targeting older hardware, so I want to have crazy good performance on my test machine which is decent hardware. If the performance is only acceptable on a good machine, then it'll suck on older machines.


It's kind of a moot point now though as the problem has righted itself. I have no idea why - I think it may have been a compiler corruption of some form, as a clean rebuild returned performance to normal, and StretchRect is indeed much faster than drawing the scene twice :)

Thanks for the help guys.