Extracting LookAt Vector from View Matrix

Do you want the look at vector in world space? If so, assuming you have a left handed coordinate system you can do the following:

In pseudo-code:

Matrix viewMatrix = camera->getViewMatrix();
viewMatrix.invert();
Vector3 lookAtInViewSpace(0, 0, 1); // Forward vector (use 0, 0, -1 if you have a right-handed coordinate system)
Vector3 lookAtInWorldSpace = lookAtInViewSpace * viewMatrix; // Transform into world space.

I am not that familiar with D3DX (which you seem to be using) but I think the transformation function you want is D3DXVec3TransformCoord.

UPDATE: No sorry, I was wrong. It is D3DXVec3TransformNormal, since you are transforming a direction, not a point.

I tried the following code:

D3DXVECTOR3 getLookAt()
{
D3DXMatrixInverse(&matView, NULL, &matView);
D3DXVECTOR3 lookAtInViewSpace(0, 0, 1);
D3DXVECTOR3 lookAtInWorldSpace;
D3DXVec3TransformNormal(&lookAtInWorldSpace, &lookAtInViewSpace, &matView);
return lookAtInWorldSpace;
}


It's not working.

Just extract the vector from column 3 of the view matrix

Like Asesh said, the third column of the matrix represents the look at vector, the first and second being the right and up vectors. If m is your D3DMATRIX then your look at vector is (m._13, m._23, m._33) in view space.

Tried the following:
D3DXMATRIX matView = camera->viewMatrix();
D3DXVECTOR3 look(matView._13, matView._23, matView._33);

Not working as it should.

I even tried:
D3DXMatrixLookAtLH(&matView,
&m_vPosition,
&m_vLook,
&m_vUp);

D3DXVECTOR3 lookResult(matView._13, matView._23, matView._33); // Extract m_vLook from the above view matrix
// Invalid result, lookResult is not equal to m_vLook


How is it not working and how do think it should be working?

D3DMatrixLookAtLH is not doing what you think it does. It computes a view matrix based on an eye position, a point to look at and an up direction. The direction you look in is based on the eye position and look at position, so your m_vLook will generally not be the same as the direction you're looking in. m_vLook is a position, not a direction.

Have a look here and here for some more information.

Edit: maybe there's some confusion about what you're trying to accomplish, are you looking for the direction you're looking in? What are you trying to accomplish?

Like Asesh said, the third column of the matrix represents the look at vector, the first and second being the right and up vectors. If m is your D3DMATRIX then your look at vector is (m._13, m._23, m._33) in view space.

To be complete, the first column could actually be a left vector, depending on choice of coordinate systems.

@Mussi: I'm looking to get the vector the camera is looking at, I'm trying to extract that vector from view matrix.

To be complete, the first column could actually be a left vector, depending on choice of coordinate systems.

Correct, it's actually left for DirectX(left handed) and right for OpenGL(right handed) by default.

@Mussi: I'm looking to get the vector the camera is looking at, I'm trying to extract that vector from view matrix.

By that you mean the direction? If so, the solution I posted in my first post is what you're looking for. What are you trying to do and what results are you getting?

I believe he is looking for the original m_vLook argument you pass as the third argument to D3DXMatrixLookAtLH().

Unfortunately, that information is lost since the third column in the view matrix is vLook - vPosition normalized. You only know the line along which the original LookAt point lies.

Let me make things more clear, I'm trying to rotate a capsule mesh to make it look at the same direction as the camera, the capsule mesh is the player.

So, when I use D3DXMatrixLookAtLH() for view matrix calculation, I can do something like the following to make the capsule look at the same direction the camera is looking at:
capsuleMesh->LookAt(m_vLook.x, m_vLook.y, m_vLook.z);

It works.

But when I don't use D3DXMatrixLookAtLH() for matrix calculation and use the following code instead:

D3DXMATRIX m_mView;
D3DXMatrixIdentity( &m_mView );
D3DXVec3Normalize( &m_vLookAt, &m_vLookAt );
D3DXVec3Cross( &m_vRight, &m_vUp, &m_vLookAt );
D3DXVec3Normalize( &m_vRight, &m_vRight );
D3DXVec3Cross( &m_vUp, &m_vLookAt, &m_vRight );
D3DXVec3Normalize( &m_vUp, &m_vUp );
m_mView._11 = m_vRight.x;
m_mView._12 = m_vUp.x;
m_mView._13 = m_vLookAt.x;
m_mView._14 = 0.0f;
m_mView._21 = m_vRight.y;
m_mView._22 = m_vUp.y;
m_mView._23 = m_vLookAt.y;
m_mView._24 = 0.0f;
m_mView._31 = m_vRight.z;
m_mView._32 = m_vUp.z;
m_mView._33 = m_vLookAt.z;
m_mView._34 = 0.0f;
m_mView._41 = -D3DXVec3Dot( &position, &m_vRight );
m_mView._42 = -D3DXVec3Dot( &position, &m_vUp );
m_mView._43 = -D3DXVec3Dot( &position, &m_vLookAt );
m_mView._44 = 1.0f;


And then try to make the capsule look at the same direction as the camera:
capsuleMesh->LookAt(m_vLook.x, m_vLook.y, m_vLook.z);
It doesn't work.

replace the third line with: D3DXVec3Normalize( &m_vLookAt, &(m_vLookAt - m_vPosition) );

Okay, let me start by saying that your mixed use of 'look at' and 'look' is confusing. Use look for directions and look at for positions or distinguish them in some other way.

This link I posted before, shows how the view matrix is built. You want to build the view matrix with the look direction, you get this by subtracting the eye position from what point you're looking at and then normalizing that. Basically, what eppo said.

I tried changing the line:
D3DXVec3Normalize( &m_vLookAt, &m_vLookAt );

To:
D3DXVec3Normalize( &m_vLookAt, &(m_vLookAt - m_vPosition) );

It mess up with the camera.

Okay, now what? You're going to have to provide a bit more information than it messes up the camera.

It's making the camera in a position like the player is dead.

Here are the default values that I use to initialize the camera:

D3DXVECTOR3 m_vPosition(0.0f, 40.0f, 10.0f)
D3DXVECTOR3 m_vLook(0.0f, 0.0f, 1.0f)
D3DXVECTOR3 m_vUp(0.0f, 1.0f, 0.0f);
D3DXVECTOR3 m_vRight(-1.0f, 0.0f, 0.0f);

Use the following code :

D3DXMATRIX m_mView;
D3DXMatrixIdentity( &m_mView );
D3DXVECTOR3 vLook = m_vLookAt;
D3DXVec3Normalize( &vLook, &(vLook - m_vPosition) );
D3DXVec3Cross( &m_vRight, &m_vUp, &vLook );
D3DXVec3Normalize( &m_vRight, &m_vRight );
D3DXVec3Cross( &m_vUp, &vLook, &m_vRight );
D3DXVec3Normalize( &m_vUp, &m_vUp );
m_mView._11 = m_vRight.x;
m_mView._12 = m_vUp.x;
m_mView._13 = vLook.x;
m_mView._14 = 0.0f;
m_mView._21 = m_vRight.y;
m_mView._22 = m_vUp.y;
m_mView._23 = vLook.y;
m_mView._24 = 0.0f;
m_mView._31 = m_vRight.z;
m_mView._32 = m_vUp.z;
m_mView._33 = vLook.z;
m_mView._34 = 0.0f;
m_mView._41 = -D3DXVec3Dot( &position, &m_vRight );
m_mView._42 = -D3DXVec3Dot( &position, &m_vUp );
m_mView._43 = -D3DXVec3Dot( &position, &vLook );
m_mView._44 = 1.0f;

@Alin: Tried that, the camera is up side down.