Depth stencil state issue

Hi eltharynd!

If you don't need the stencil test, you probably want to disable it.

dsDesc.StencilEnabled = false;

In this case, you can put full precision into the depth value by using a full float for it:

depthTexDesc.Format = DXGI_FORMAT_D32_FLOAT;

Currently, your depth testing function permits all fragments to be written. Instead, let only fragments through that have a smaller depth value.

dsDesc.DepthFunc = D3D11_COMPARISON_LESS_EQUAL;

It might be just due to copy-pasting into the browser, but did you use the correct depth stencil view description, when creating the depth stencil view? (2nd argument changed)

mDevice->CreateDepthStencilView(depthStencilTexture, &depthStencilViewDesc, &mDepthStencilView);

I guess you did, but for completeness I'm just saying it: You bound your depth buffer like so, right?

ID3D11RenderTargetView* rtvs[] = { mRenderTargetView_Backbuffer };
ImmediateContext->OMSetRenderTargets(1, rtvs, mDepthStencilView);

Do you clear the depth stencil view before drawing?

mImmediateContext->ClearDepthStencilView(mDepthStencilView, D3D11_CLEAR_DEPTH, 1, 0);

Also, were all your resources successfully created? (That is, they are not NULL?)

Best, Tsus

- dsDesc.DepthFunc needs to be D3D11_COMPARISON_LESS.

- make sure you have set the correct culling mode in the rasterizer state (seems alright).

- make sure you bind the depth-stencil surface with OMSetRenderTargets().

- dsDesc.StencilEnable can be set to false, unless you have some reason to use stencil-ops.

- enable the debug-layer.

thanks everyone for the replies....

dsDesc.DepthFunc was less before i changed everything then i forgot to put it back :/

Tsus you were right i was giving the wrong depth stencil view description ( although it probably was due to the copy paste as you said)

i tried to change the format

i was already using OMsetblablabla()

the current code is this

ID3D11Texture2D* depthStencilTexture;

	D3D11_TEXTURE2D_DESC depthTexDesc;
	ZeroMemory (&depthTexDesc, sizeof(D3D11_TEXTURE2D_DESC));
	depthTexDesc.Width = set->mapSettings["SCREEN_WIDTH"];
	depthTexDesc.Height = set->mapSettings["SCREEN_HEIGHT"];
	depthTexDesc.MipLevels = 1;
	depthTexDesc.ArraySize = 1;
	depthTexDesc.Format = DXGI_FORMAT_D32_FLOAT;
	depthTexDesc.SampleDesc.Count = 1;
	depthTexDesc.SampleDesc.Quality = 0;
	depthTexDesc.Usage = D3D11_USAGE_DEFAULT;
	depthTexDesc.BindFlags = D3D11_BIND_DEPTH_STENCIL;
	depthTexDesc.CPUAccessFlags = D3D11_CPU_ACCESS_WRITE | D3D11_CPU_ACCESS_READ;
	depthTexDesc.MiscFlags = 0;

	mDevice->CreateTexture2D(&depthTexDesc, NULL, &depthStencilTexture);

	D3D11_DEPTH_STENCIL_DESC dsDesc;

	// Depth test parameters
	dsDesc.DepthEnable = true;
	dsDesc.DepthWriteMask = D3D11_DEPTH_WRITE_MASK_ALL;
	dsDesc.DepthFunc = D3D11_COMPARISON_LESS_EQUAL;//LESS

	// Stencil test parameters
	dsDesc.StencilEnable = false;
	dsDesc.StencilReadMask = 0xFF;
	dsDesc.StencilWriteMask = 0xFF;

	// Stencil operations if pixel is front-facing
	dsDesc.FrontFace.StencilFailOp = D3D11_STENCIL_OP_KEEP; //KEEP
	dsDesc.FrontFace.StencilDepthFailOp = D3D11_STENCIL_OP_INCR; //INCR
	dsDesc.FrontFace.StencilPassOp = D3D11_STENCIL_OP_KEEP; //KEEP
	dsDesc.FrontFace.StencilFunc = D3D11_COMPARISON_ALWAYS;

	// Stencil operations if pixel is back-facing
	dsDesc.BackFace.StencilFailOp = D3D11_STENCIL_OP_KEEP; //KEEP
	dsDesc.BackFace.StencilDepthFailOp = D3D11_STENCIL_OP_DECR; //DECR
	dsDesc.BackFace.StencilPassOp = D3D11_STENCIL_OP_KEEP; //KEEP
	dsDesc.BackFace.StencilFunc = D3D11_COMPARISON_ALWAYS;

	// Create depth stencil state
	mDevice->CreateDepthStencilState(&dsDesc, &mDepthStencilState);

	mDeviceContext->OMSetDepthStencilState(mDepthStencilState, 1);
	
	D3D11_DEPTH_STENCIL_VIEW_DESC depthStencilViewDesc;
	ZeroMemory (&depthStencilViewDesc, sizeof(depthStencilViewDesc));
	depthStencilViewDesc.Format = depthTexDesc.Format;
	depthStencilViewDesc.ViewDimension = D3D11_DSV_DIMENSION_TEXTURE2D;
	depthStencilViewDesc.Texture2D.MipSlice = 0;

	mDevice->CreateDepthStencilView(depthStencilTexture, &depthStencilViewDesc, &mDepthStencilView);

and later i pass them at the OM

mDeviceContext->OMSetRenderTargets(1, &mTargetView, mDepthStencilView);

and of course before every frame i call

void Engine::ClearTargetView()
{
mDeviceContext->ClearRenderTargetView(mTargetView, D3DXCOLOR(0.0f, 0.0f, 0.0f, 1.0f));
mDeviceContext->ClearDepthStencilView(mDepthStencilView, D3D11_CLEAR_DEPTH, 1.0f, 0 );
}

note that for some reason

if i set depthTexDesc.CPUAccessFlags = 0;

nothing shows on the screen.....

if i set it to cpu access read and write i can see my models but i get the same error with the depth test....

i don't think the cpu ever tries to access the depth buffer to be honest..

I am really lost at this point...

PS... i think it's irrelevant but i didn't set the resterizer yet....

it should not matter since it uses a default one.. right?

Hi eltharynd,

Usually, CPU access flags are not required for the depth buffer.

Try to turn on the debug layer, as eppo suggested. D3D will most probably tell you what is wrong. The behavior you described sounds like invalid parameters. Pass the flag D3D11_CREATE_DEVICE_DEBUG into the Flags parameter of the D3D11CreateDevice call.

Could you check that all your mDevice->Create... methods return S_OK?
If not, the debug layer will tell you in the Output window of visual studio, why the Create-method failed.
(If you run in Debug mode with F5, not Ctrl+F5.)

Clearing structs with ZeroMemory is a good practice (in my eyes). Could you do it for the depth stencil desc, too? (Just to be safe.)

ZeroMemory (&dsDesc, sizeof(D3D11_DEPTH_STENCIL_DESC));

Also, you have some copy paste error when clearing the depth stencil view desc. It should go:

ZeroMemory (&depthStencilViewDesc, sizeof(D3D11_DEPTH_STENCIL_VIEW_DESC));

Just to be safe, set a rasterizer state. Whenever you move code around, you don't want to depend on states of some previous code block.

D3D11_RASTERIZER_DESC rsDesc;
ZeroMemory(&rsDesc, sizeof(D3D11_RASTERIZER_DESC));
rsDesc.CullMode = D3D11_CULL_BACK;
rsDesc.DepthBias = 0;
rsDesc.DepthBiasClamp = 0;
rsDesc.FillMode = D3D11_FILL_SOLID;
rsDesc.AntialiasedLineEnable = false;
rsDesc.DepthClipEnable = true;
rsDesc.FrontCounterClockwise = true;
rsDesc.MultisampleEnable = true;
rsDesc.ScissorEnable = false;
rsDesc.SlopeScaledDepthBias = 0;
if (FAILED(mDevice->CreateRasterizerState(&rsDesc, &mRsBackfaceCulling))) return false;

Then, later:

mDeviceContext->RSSetState(mRsBackfaceCulling);

And don't worry. You'll figure it out.

Best, Tsus

ok so if i set CPU access flags to write/read i get E_INVALIDARG

so i set them back to 0 and i get everything S_OK but then nothing is rendered...

Ok. So, with all resources being successfully created and no errors in the output window during draw calls, it's time to use a graphics debugger like PIX.
PIX is contained in the DxSDK and allows you to capture the draw calls and states for a frame (every time you hit F12).
When clicking on a draw call in the list presented to you in PIX, the vertices of the triangles are shown before and after transformation.
If you do a right click on the rendering output you can debug individual pixels and see, why fragments were discarded.

Also, you can dig into the states bound at a draw call.
Hopefully, this sheds some light on the matter.

ok so if i set CPU access flags to write/read i get E_INVALIDARG

so i set them back to 0 and i get everything S_OK but then nothing is rendered...

If you want to use a resource with cpu access, you need to set the usage to D3D11_USAGE_DYNAMIC instead of D3D11_USAGE_DEFAULT. Did you do that?

i tried both actually cause i really have no clues about what's wrong...

anyway now i tried to run PIX (even tho im not 100% sure about how to use it but i get this results:

so if i understand this correctly my mesh is there as i can see from RGB channel but if i change to depth channel everything is blacked out.... so what could this mean?

Hi!

The black depth channel is somewhat expected, due to the non-linear distribution of the z-values. Right beside the combobox where you selected the "depth" channel, you have a gray bar that allows you to filter the range displayed linearly. There are tiny triangles (top-left at the bar, and bottom-right). These are the lower and upper bounds. You can move them around, so that you have the lower bound around 0.95. That should show you something.

On the top left of the window is a small "save" icon. You could save the PIXrun and upload it, so that we might have a look.

Best, Tsus

Additionally, click on the device context "hyperlink" (it only turns blue after you have looked at the render or mesh tab). You can now inspect what states are set and what resources are bound, one tab for every pipeline stage.

Lucky guess: Depth doesn't work if the render target and depth buffer have a mismatching size or sample description, but that should turn up from the debug layer (in PIX it's the "Output Log" tab). Also check your projection matrix: What near plane do you use ?

https://docs.google.com/file/d/0B7G-_39cTN3nSUFyYldITk1VUlU/edit?usp=sharing

here'the run.....thanks everybody for the help by the way...

P.S this is my projection matrix

mProj = XMMatrixPerspectiveFovLH( 1.0, set->mapSettings["SCREEN_WIDTH"] / (FLOAT)set->mapSettings["SCREEN_HEIGHT"], 0.01f, 100.0f );

and viewport just in case

void Engine::InitViewport()
{
	ZeroMemory (&mViewport, sizeof(D3D11_VIEWPORT));
	//set the viewport
	mViewport.TopLeftX = 0;
	mViewport.TopLeftY = 0;
	mViewport.Width = set->mapSettings["SCREEN_WIDTH"];
	mViewport.Height = set->mapSettings["SCREEN_HEIGHT"];
	mViewport.MinDepth = 0.0f;
    mViewport.MaxDepth = 1.0f;
	SetViewport();
}

void Engine::SetViewport()
{
	mDeviceContext -> RSSetViewports(1, &mViewport);
}

These are the lower and upper bounds. You can move them around, so that you have the lower bound around 0.95. That should show you something.

actually it doesn't :) i tried different values but it's still either full black or full white...

kind of expectable since it's not working in the first place :/

Lucky guess: Depth doesn't work if the render target and depth buffer have a mismatching size or sample description, but that should turn up from the debug layer (in PIX it's the "Output Log" tab). Also check your projection matrix: What near plane do you use ?

this might be it actually.. .i never set a render target view description...... I'm coding it right now to see if it helps....

EDIT:

http://msdn.microsoft.com/en-us/library/windows/desktop/ff476201(v=vs.85).aspx

there doesn't seem to be a sample description...

ok i probably am missing something on how to add a target description as it seems to work differently than for other resources...

ID3D11Texture2D* pTexture;
	mSwapChain -> GetBuffer(0, __uuidof(ID3D11Texture2D), (LPVOID*)&pTexture);

	D3D11_TEXTURE2D_DESC textureDesc;
	textureDesc.Width = set->mapSettings["SCREEN_WIDTH"];
	textureDesc.Height = set->mapSettings["SCREEN_HEIGHT"];
	textureDesc.MipLevels = 1;
	textureDesc.ArraySize = 1;
	textureDesc.Format = DXGI_FORMAT_R32G32B32A32_FLOAT;
	textureDesc.Usage = D3D11_USAGE_DEFAULT;
	textureDesc.BindFlags = D3D11_BIND_RENDER_TARGET;
	textureDesc.SampleDesc.Count=1;
	textureDesc.SampleDesc.Quality=0;
	textureDesc.CPUAccessFlags = 0;
	textureDesc.MiscFlags = 0;

	HRESULT hr = mDevice->CreateTexture2D(&textureDesc, NULL, &pTexture);

	D3D11_RENDER_TARGET_VIEW_DESC targetViewDescription;
	ZeroMemory(&targetViewDescription, sizeof(D3D11_RENDER_TARGET_VIEW_DESC));
	targetViewDescription.Format = DXGI_FORMAT_D32_FLOAT;

	hr =mDevice ->CreateRenderTargetView(pTexture, &targetViewDescription, &mTargetView);
	pTexture -> Release();
	pTexture = 0;

first hr is ok and second invalidarg

Now you made it worse (by adding to the confusion). Take a step back. You already had a render target view (mRenderTargetView_Backbuffer) set otherwise nothing would render. So your PIX run was "fine", and I think I was spot on: Your bound render target texture (probably coming from your swap chain) has is 1304x478 and your depth buffer is 1280x720.

Normally I would suspect the common pitfall of a mismatch between windows client size and render target size (search the forum and also lookup GetClientRect and AdjustWindowRectEx in the WinAPI docs).

But here the height is so far off, there's likely something else going on. Anyway: render target and depth size must match (forget about sample description for now, this is only for MSAA, just use (1,0) throughout).

It looks like you're getting your dimensions from a (global) configuration. Make sure those values are correct after window and swapchain creation (and after window resize, if you handle that too).

Edit: Wait, PIX is making a mess on my side with that run. The swapchain texture changes size dependant of the render tab size (huh?). In the objects tab the sizes match perfectly 1280x720. Strange. Depth seems to work, by the way.

Edit: Oh, yeah: You really should use the D3D11_CREATE_DEVICE_DEBUG flag. In that PIX run you still don't. As said, the debug layer would scream about the size mismatch.

ok so i noticed i was using different samples for the swapchain and the depth buffer...

now if i set them both to 1,0 i can see my meshes but not the console..... if i set them to 8,1 i can see them both.. does someone have an explanation for this?

the depth buffer is not working anyway...

It's working.... ok turns out i forgot to set the swapchain width and height.....

so i set them same way i set everything else and it's now working correctly...