DOF basic steps

Several ways, the most simple(but maybe not best) would be:

1.- Somehow, calculate at which distance you focus your eye. For example, shoot a ray at the center of your screen and measure how far it comes. Not a superb way, but good for a start.
2.- Render the depth of your scene into a texture (depth = distance between camera and pixel).
This buffer is useful for tons of others effects as well btw.

In this buffer, you can also calculate how far away the depth is from your focus distance. If equal, no blur. If closer or further, increase the value. Check out "Circle for confusion" if you want some background.


3.- Render the scene as usual
4.- Capture the rendered scene in a textureA
5.- Blur textureA (you can do this on a lower resolution eventually). Eventually use the distance-factor from step 2 to determine the blur-size (pick X samples in a certain radius around the center pixel). Store results in textureB
6.- As a post effect, mix between the blurred and unblurred texture based on the depth or factor you calculated in step 2.

Something like that.

Masaki Kawase presents a framebuffer-based DOF effect:

http://www.daionet.g...2003_DSTEAL.ppt

I would actually use a near and far plane for the effect of the depth of field, technically it's not as realistic as using the single depth value, but it will allow you to set a range within which everything will be rendered sharp.

So you have your two values that are determined by a raytrace and some logic, or in cutscenes it's controlled by the camera animation; then you use that as your sharp range.

If you use even more values you can set the distance over which blurriness increases.

What you need:
1. What I've said above
2. A render of the depth of the scene, so stencil, or render Z to target. You can use a z depth and then figure out the bluriness from that, or you can just figure out the bluriness of a portion in this fragment shader and output it, (probably not if you're using stencil buffer, I'm actually not sure if stencil can be used, I've only ever used a render target for DOF)
2.a. If you don't want the skybox to be blurred, then have a fragment shader that just outputs a value that is between your sharp range, specifically for your skybox
3. Your scene rendered normally, sans UI, to a target
3.a. Do some experimentation as to what you want blurred here, maybe you don't want to blur special effects like volumetric lighting, lens flares, or bullets, maybe you want to blur geometry that is through a window and not the window itself, etc, so the geometry that is through the window might be in your sharp range when the window is not, and that will lead to that geometry being blurry because of the window's depth being used. Maybe as a rule just ignore transparent, semi-transparent objects in your depth rendering.
3.b. But keep in mind that when you compose the final image to the graphics card's back buffer, you will not have a depth buffer of the geometry rendered here to help you if you use standard rendering to render them, you will have the depth buffer from 2, but it isn't the same thing, early z rejection is a powerful performance tool
4. Render your target from 3 to the backbuffer using the z depth from 2 to determine blur.

Sorry that I blurred 2 and 3 together.... but you should be able to figure it out.

I'm still learning stencil stuff myself, but it might be possible to do 2 with your render target depth buffer, while also rendering stencil, then some how you stencil as your early z rejection for 3 to reduce the amount of pixels you have to draw significantly.

If the stencil buffer doesn't work like that, and as I said it's a hole in my knowledge, and works as a 'stencil' or more accurately described a 'mask' that tells the GPU to skip any pixels that don't fall within it's mask, then you can use the stencil buffer for a cheap and dirty DOF. Here's how, if the stencil buffer is a mask, so it only has 2 colors that can be written to it
1. Generate two values only, for sharp start and end, you have no gradual effect with this technique
2. Render your scene geometry to a target, write to the stencil buffer as if it were a mask, using one color for pixels that are within the DOF sharp range, and a different color for the ones that aren't
3. Render the result of your target to the backbuffer with a fragment shader that does not blur, so pure pixel copy, with stencil mask enabled so that only the pixels that are within the DOF sharp range as masked by the stencil buffer are rendered.
4. Render the result of your target to the backbuffer with a fragment shader that does blur, also pure pixel copy aswell, with a stencil mask enabled (and flipped from 3) so that only the pixels that are not within the DOF sharp range as masked by the stencil buffer are rendered.

Pretty cheap and dirty that, probably won't look great, but limits the amount of actual geometry rendering you have to do to 1 pass. I don't know if you can grab a stencil from a render target though, seems it's the same limitation as the depth buffer in that you can't get it from a render target for your backbuffer, making this cheap and dirty technique impossible. Why can I no has depth buffer dammit?

Take a look at "Chapter 23. Depth of Field: A Survey of Techniques" from GPU Gems 3 and see if that helps.