Reverse mouse picking

I'm making a 3D game with an overhead freeranging camera. I have a method to mouse pick that works correctly, creating a ray for the mouse point then projecting it til it collides with something. I know it works properly because any markers I create show up at the right places.

Several times I've tried to go the other direction, though, and always had problems. Using the rays and camera focus and all that crap is very error prone. I don't even know for sure all of it works correctly in the engine. Most likely it does but there's a chance I am using something wrong or whatever. At any rate, i've given up on the recommended method it just doesn't seem to work.

Instead I want to just compute it using geometry. However, that seems to be failing too, but hopefully it's failing in a way that someone else can point out.

So using that method to project into the game world, I am projecting out from each of the corner mouse positions into the plane where z = 0. That is projecting from 3D perspective to a 2D plane.

From there I take the x and y axes, and call a method to find the closest point on each axis to the reverse pick point. Then I find the distance from the origin and use this to calculate the mouse cursor location for that point. Seems like it should work, but somehow it fails. Does this approach have something wrong with it? If not I guess there can be a bug somewhere else. Now the dummy case should be to project the cursor out to the plane, then project it back but it always fails and I can't see what's wrong. In practice there's a reason for doing this that can't be ignored, though, that's just the test case.

If I print out everything it seems if I project every mouse point to the plane the distances between them are regular and describe lines as I'd expect them to. So it seems like my method should be workable but it completely fails. I know the plane can be skewed but (or so I think) it should not matter.

Here is the code. Maybe something obvious is there? Been screwing with this 14 hours today so at this point a fresh pair of eyes would be helpful.



//first, I have this method which I know works
Point3 CollisionUtil::projectCursorPointToGround(Point screenPoint)


//this method fails
Point CollisionUtil::screenPointForWorldPoint(Point3 worldPoint)
{
Point cursorPoint;

float screenWidth = getDisplayWidth();
float screenHeight = getDisplayHeight();


Point3 last = projectCursorPointToGround(Point(0, 0));

Point3 ul = projectCursorPointToGround(Point(0, 0));
Point3 ur = projectCursorPointToGround(Point(screenWidth, 0));
Point3 ll = projectCursorPointToGround(Point(0, screenHeight));
Point3 lr = projectCursorPointToGround(Point(screenWidth, screenHeight));

Point3 xAxis = ur-ul;
Point3 yAxis = ll-ul;

Point3 pointOnXAxis = closestPointOnLine(worldPoint, ul, ur);
Point3 pointOnYAxis = closestPointOnLine(worldPoint, ul, ll);


Point3 xProjection = pointOnXAxis-ul;
Point3 yProjection = pointOnYAxis-ul;


float xAxisMagnitude = magnitude(xAxis);
float yAxisMagnitude = magnitude(yAxis);

float xRatio = screenWidth/xAxisMagnitude;
float yRatio = screenHeight/yAxisMagnitude;

float xMagnitude = magnitude(xProjection);
float yMagnitude = magnitude(yProjection);

long pointX = round(xMagnitude*xRatio);
long pointY = round(yMagnitude*yRatio);

cursorPoint.x=pointX;
cursorPoint.y=pointY;

return cursorPoint;
}