Hello!
I'm trying to perform backface culling in a geometry shader. So I'm sending a point as a vec3 from the cpu to the gpu through a vbo and then I generate a cube relative to that point in the shader, and I only wish to generate the sides that are acually visible, so I must test if each side is visible to the camera. What I have right now kind of works, but the faces cull too early.
I'm using the dot product technique where I take the difference between the camera position and the position of one vertex on the side I wish to test against and dot it with the sides normal.
vec3 point = gl_in[0].gl_Position.xyz + vec3(-0.5f,-0.5f,-0.5f);
vec3 diff = point - CameraPosition[0];
float k = dot(diff, vec3(-1.0f, 0.0f, 0.0f));
if (k <= 0.0f) {
..visible
}
That's how I do it in my shader, the gl_in[0].gl_Position lies in the center of the cube. The cube is not rotated (but the world is when the camera rotates) so I assumed that the normal for the left side would just be (-1,0,0).
I hope I've given enough information to give you an idea what might cause the problem.
-Christopher.