I was wondering if anyone could explain the depth buffer and the depth stencil state comparison function to me as I'm a little confused

So I have set up a depth stencil state where the DepthFunc is set to D3D11_COMPARISON_LESS, but what am I actually comparing here? What is actually written to the buffer, the pixel that should show up in the front?

I have these 2 quad faces, a Red Face and a Blue Face. The Blue Face is further away from the Viewer with a Z index value of -100.0f. Where the Red Face is close to the Viewer with a Z index value of 0.0f.

When DepthFunc is set to D3D11_COMPARISON_LESS the Red Face shows up in front of the Blue Face like it should based on the Z index values. BUT if I change the DepthFunc to D3D11_COMPARISON_LESS_EQUAL the Blue Face shows in front of the Red Face. Which does not make sense to me, I would think that when the function is set to D3D11_COMPARISON_LESS_EQUAL the Red Face would still show up in front of the Blue Face as the Z index for the Red Face is still closer to the viewer

Am I thinking of this comparison function all wrong?

Vertex data just in case

//Vertex date that make up the 2 faces
Vertex verts[] = {

  		//Red face
		Vertex(Vector4(0.0f, 0.0f,     0.0f), Color(1.0f, 0.0f, 0.0f)),
		Vertex(Vector4(100.0f, 100.0f, 0.0f), Color(1.0f, 0.0f, 0.0f)),
		Vertex(Vector4(100.0f, 0.0f,   0.0f), Color(1.0f, 0.0f, 0.0f)),
		Vertex(Vector4(0.0f, 0.0f,     0.0f), Color(1.0f, 0.0f, 0.0f)),
		Vertex(Vector4(0.0f, 100.0f,   0.0f), Color(1.0f, 0.0f, 0.0f)),
		Vertex(Vector4(100.0f, 100.0f, 0.0f), Color(1.0f, 0.0f, 0.0f)),
  		
  		//Blue face
		Vertex(Vector4(0.0f, 0.0f,     -100.0f), Color(0.0f, 0.0f, 1.0f)),
		Vertex(Vector4(100.0f, 100.0f, -100.0f), Color(0.0f, 0.0f, 1.0f)),
		Vertex(Vector4(100.0f, 0.0f,   -100.0f), Color(0.0f, 0.0f, 1.0f)),
		Vertex(Vector4(0.0f, 0.0f,     -100.0f), Color(0.0f, 0.0f, 1.0f)),
		Vertex(Vector4(0.0f, 100.0f,   -100.0f), Color(0.0f, 0.0f, 1.0f)),
		Vertex(Vector4(100.0f, 100.0f, -100.0f), Color(0.0f, 0.0f, 1.0f)),
	};

Screen space (post-projection / NDC) Z goes from 0 to 1 (or 0 to w in post-projection). Your vertex shader transforms your vertex data to this space, usually using a projection matrix.

Maybe your projection matrix is squashing your Z range so that both triangles end up at NDCz=0

On 11/18/2017 at 1:33 AM, noodleBowl said:

but what am I actually comparing here?

You're comparing the value thats already in the depth buffer to the z value of the fragment you want to write to the screen.

edit - did I get that backwards...

9 hours ago, Infinisearch said:

You're comparing the value thats already in the depth buffer to the z value of the fragment you want to write to the screen.

This is what I thought, which was why I was really confused. Turns out my problem had nothing to do with my depth buffer/depth stencil state. Turns out the viewport has MinDepth and MaxDepth properties. I never set these values for my viewport after setting these everything seems to work correctly depth wise

Side question about the Z near and Z far plans for projection matrices. If I have a right handed coord system, so positive Z points towards the Viewer (or me) and negative Z goes further into the screen does this mean my Z near plan should be a positive value and my Z far plan should be negative value? Does this change between different projection matrix types such as Perspective vs Orthographic?

That depends on your function or library that you're using for creating your project matrix. If you're using D3DX or DirectXMath functions for creating a "right-handed" perspective projection matrix, then you'll specify your near and far clipping plane parameters as the absolute distance from the camera to the plane. So they would both be positive, typically with zFar > zNear (although you can flip them if you want a reversed Z buffer, which can be be useful for working with floating point depth buffer formats). This is the same for the orthographic projections in both of those libraries. If you're using a different library, you should check the documentation to see what values they expect, although it's very likely to be the same as the D3DX/DirectXMath functions.

FYI the documentation for the D3DX matrix functions can be helpful to look at, since they show you exactly how the matrix is constructed (you can also look at the actual implementations for the DirectXMath functions, since it's all in inline header files). If  you look at the RH projection functions, you'll see that they essentially end up negating the Z value so that it ends up positive in clip space, since this is what D3D expects.