Skip to main content
gamedev.net
🔒 Locked

D3DXIntersect() problem

Started by durban 3 replies 1.3k views
Original Post
durban
durban
I'm simply trying to be able to pick an object with the mouse and change its render properties when its clicked. Problem : The results aren't correct, the mesh seems to be moved over. When I click the mesh nothing happens, however, if I click in the region that D3DXIntersect "thinks" the mesh is, I get the proper result. Anyone see a problem in this code? : Pick() - returns true if the user clicked the mesh : 

bool CPick::Pick( LPDIRECT3DDEVICE9 device, LPD3DXMESH mesh, D3DXMATRIX matProj, D3DXMATRIX matView, HWND hWnd )
{
	D3DXVECTOR3 vecRay, vecDir, v;
	D3DXMATRIX m;

	D3DXMatrixInverse(&m, NULL, &matView);

	POINT ptCursor;
	GetCursorPos( &ptCursor );
	ScreenToClient( hWnd, &ptCursor );

	v.x = (((2.0f * ptCursor.x) / 400) - 1) / matProj._11;
	v.y = (((2.0f * ptCursor.y) / 300) - 1) / matProj._22;
	v.z = 1.0f;

	vecRay.x = m._41;
	vecRay.y = m._42;
	vecRay.z = m._43;
	vecDir.x = v.x*m._11 + v.y*m._21 + v.z*m._31;
	vecDir.y = v.x*m._12 + v.y*m._22 + v.z*m._32;
	vecDir.z = v.x*m._13 + v.y*m._23 + v.z*m._33;


	BOOL Hit;
	DWORD Face;
	float u, w, Dist;

	D3DXIntersect(mesh, &vecRay, &vecDir, &Hit, &Face, &u, &w, &Dist, NULL, NULL);
	
	if( Hit )
		return true;
	else
		return false;
}
Code Snippet from the Render() function where the call to Pick() is made : 

if(drawMesh == TRUE )
{
	sMesh* temp = m_CObj->GetStartPtr();
	HWND hwnd;
	if( hwnd = GetCapture())
	{
		do
		{
			if( m_CPick->Pick( _device, temp->_mesh, matProj, matView, hwnd))
			{
				temp->bHit = TRUE;
				MessageBox(GetMainWndHandle(), "bHit=TRUE", "",MB_OK);
				break;
			}
			temp = temp->Next;
		}while( temp->Next != NULL );
			
		temp = m_CObj->GetStartPtr();
	}

	do
	{
		if( temp->bHit == TRUE )
		{
			_device->SetMaterial(&temp->_materials[0]);
			_device->SetRenderState(D3DRS_FILLMODE, D3DFILL_SOLID);
			_device->SetRenderState(D3DRS_AMBIENT, 0x00FF00FF);
			temp->_mesh->DrawSubset(0);
			_device->SetRenderState(D3DRS_FILLMODE, D3DFILL_WIREFRAME);
			_device->SetRenderState(D3DRS_AMBIENT, 0x00000000);
		}
		else
		{
			for( DWORD i=0; i<temp->_numMaterials; i++)
			{
				_device->SetMaterial(&temp->_materials);
				_device->SetTexture(0, temp->_textures);

				temp->_mesh->DrawSubset(i);
			}
		}
		temp = temp->Next;
	}while( temp->Next != NULL );

	temp = m_CObj->GetStartPtr();			
}
The World/View/Projection matrices were defined earlier in the Render() function* I will appreciate any help/comments given, Thanks in advance
neneboricua19
neneboricua19
You must transform your ray into the model space of your mesh. Typically, this is done by taking the inverse of the world matrix you use to position your model. Transform your ray's position and direction by this inverse matrix and you should be able to perform your ray-intersection test without a problem.

neneboricua
durban
durban
It's still shifted after I did that. I'm using 4 different windows to show each perspective of the mesh with swap chains but that shouldn't matter since I'm passing the window's world/view/proj matrices to the function right? Here's the code I added:

D3DXMatrixInverse(&world, NULL, &matWorld);D3DXVec3TransformCoord( &vecRay, &vecRay, &world );D3DXVec3TransformCoord( &vecDir, &vecDir, &world );


I noticed that where D3DXIntersect thinks the mesh is, is actually a reflection of the mesh on the -y axis. I'm still stumped, any ideas?
neneboricua19
neneboricua19
You need to transform the starting point of the ray using D3DXVec3TransformCoord, but the ray direction should be transformed using D3DXVec3TransformNormal. The reason for this is that each function assumes a different value for the "w" component of the vector.

neneboricua
durban
durban
Quote:
Original post by neneboricua19
You need to transform the starting point of the ray using D3DXVec3TransformCoord, but the ray direction should be transformed using D3DXVec3TransformNormal. The reason for this is that each function assumes a different value for the "w" component of the vector.

neneboricua


Great, it works like a charm now. After doing what you said I noticed where D3DXIntersect thought the mesh was, was actually the mesh reflected upon the -y axis. so I simply changed

v.y = (((2.0f * ptCursor.y) / 300) - 1) / matProj._22;
to
v.y = -(((2.0f * ptCursor.y) / 300) - 1) / matProj._22;

Thanks alot for the help man :)

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.