Objects appear inverted and deformed when camera rotates 180° in my software renderer

Hi everyone,

I’m building a 3D renderer from scratch in C (no OpenGL/DirectX). I’ve implemented my own camera system with yaw and pitch, and a simple “look-at” style view matrix using mat4_pointAt() and a quick inverse for the view matrix. Everything works fine — camera rotation, panning, and projection — except for one issue:

When I rotate the camera around 180° (to look backward), my rendered object suddenly looks inverted/back-to-front and slightly deformed.

When I continue rotating, it goes back to normal again.

Here’s my camera creation and update code (simplified):

void InitCamera(vec3 pos,float yaw,float pitch,Camera** camera){
  *camera = (Camera*)malloc(sizeof(Camera));
  (*camera)->pitch = pitch;
  (*camera)->yaw = yaw;
  vec3 world_up = {0.0,1.0,0.0};
  vec3 cameraPos = {0.0,0.0,0.0};
  vec3 lookDir = {0.0,0.0,1.0};
  vec3 target;
  vec3_add(cameraPos,lookDir,target);
  mat4 m;
  mat4_pointAt(cameraPos,target,world_up,m);
  mat4_quickInverse(m, (*camera)->view);
}

void rotateCamera(Camera** camera)
{
  float yawRad = (*camera)->yaw * (M_PI / 180.0f);
  float pitchRad = (*camera)->pitch * (M_PI / 180.0f);
  vec3 worldUp = {0.0f, 1.0f, 0.0f};
  (*camera)->lookdir[0] = cosf(pitchRad) * sinf(yawRad);
  (*camera)->lookdir[1] = sinf(pitchRad);
  (*camera)->lookdir[2] = cosf(pitchRad) * cosf(yawRad);
  vec3_normalize((*camera)->lookdir);

  vec3 right, up;
  vec3_cross(worldUp, (*camera)->lookdir, right);
  vec3_normalize(right);
  vec3_cross((*camera)->lookdir, right, up);
  vec3_normalize(up);

  vec3 target;
  vec3_add((*camera)->pos, (*camera)->lookdir, target);
  mat4 viewMat;
  mat4_pointAt((*camera)->pos, target, up, viewMat);
  mat4_quickInverse(viewMat, (*camera)->view);
}

I also handle the camera yaw/pitch updates from mouse input:

case WM_MOUSEMOVE:
 if(mousePressed) {
   float sensitivity = 0.1;
   int mouseX = GET_X_LPARAM(lParam);
   int mouseY = GET_Y_LPARAM(lParam);
   float dyaw = mouseX - oldX;
   float dpitch = oldY - mouseY;
   dyaw *= -sensitivity;
   dpitch *= sensitivity;
   oldX = mouseX;
   oldY = mouseY;
   GCamera->yaw += dyaw;
   GCamera->pitch += dpitch;
   if(GCamera->pitch > 89.0f) GCamera->pitch = 87.0f;
   if(GCamera->pitch < -89.0f) GCamera->pitch = -87.0f;
   rotateCamera(&GCamera);
 }
 break;

The projection matrix looks like this:

void create_mat4_perscpective(float fovy, float aspect, float znear, float zfar, mat4 M) {
 float f = 1 / tanf(fovy / 2.0f);
 memset(M, 0, sizeof(mat4));
 M[0][0] = f / aspect;
 M[1][1] = f;
 M[2][2] = zfar / (zfar - znear);
 M[2][3] = 1.0f;
 M[3][2] = (-zfar * znear / (zfar - znear));
}

My question is:
Why does the object become inverted/back-to-front and deformed every time the camera rotates around 180°, and how can I make my camera orientation stable through full rotations?

Is this a handedness or basis-flip issue with my pointAt/cross product order, or something else related to the view matrix inversion?