34 minutes ago, scragglypoo said:

As for local space and world space, I have read a * b is world space, but b * a is local space so that you don't need to create axis angles?

I've never heard of this.  I guess it could work that way.  I'm going to look into this because it sounds intriguing and would save a lot of computation in my code for local rotations.  But rotation about an arbitrary axis is pretty complex.  Maybe it simplifies way down if the axis isn't arbitrary?  But anyway here's my C code for axis rotation:

matrix rotationAboutAxis(float angle, vector axis)
{
    float x2 = axis.x*axis.x;
    float y2 = axis.y*axis.y;
    float z2 = axis.z*axis.z;
    
    float a = (degToRad(angle))/2.0;
    float s = sinf(a);
    float c = cosf(a);
    float s2 = s*s;
    float sc = s*c;
    
    matrix rot = matrixIdentity();
    
    rot[0][0] = 1-2*(y2+z2)*s2;
    rot[0][1] = 2*(axis.x*axis.y*s2+axis.z*sc);
    rot[0][2] = 2*(axis.x*axis.z*s2-axis.y*sc);
    
    rot[1][0] = 2*(axis.y*axis.x*s2-axis.z*sc);
    rot[1][1] = 1-2*(z2+x2)*s2;
    rot[1][2] = 2*(axis.y*axis.z*s2+axis.x*sc);
    
    rot[2][0] = 2*(axis.z*axis.x*s2+axis.y*sc);
    rot[2][1] = 2*(axis.z*axis.y*s2-axis.x*sc);
    rot[2][2] = 1-2*(x2+y2)*s2;
    
    return rot;
}

18 hours ago, missionctrl said:

I've never heard of this.  I guess it could work that way.  I'm going to look into this because it sounds intriguing and would save a lot of computation in my code for local rotations.  But rotation about an arbitrary axis is pretty complex.  Maybe it simplifies way down if the axis isn't arbitrary?  But anyway here's my C code for axis rotation:

 

Here is where I read it; maybe it applies to only quaternions

https://forum.unity3d.com/threads/understanding-rotations-in-local-and-world-space-quaternions.153330/

Thanks! I got your code working here:

Matrix Rotate( Matrix rotation ) {
	// Rotate around local x-axis.
	auto x = RotationAboutAxis( this.right, -rotation.Pitch() );
	this = this * x;
	// Rotate around world y-axis.
	auto y = RotationAboutAxis( Vector::Up, rotation.Yaw() );
	this = this * y;
	return this;
}

Does it matter what order I put the rotations in? It works either way this*x or this*y first.

Also why does y need to be a world-space rotation? why not x?
 

Thanks for the link!  Some interesting stuff.  I think I see what's going on... it looks like changing the order performs the rotation around a locally-centered origin, but not in a local orientation. Maybe that's my fault for mis-using terminology a bit. When I say "local rotation" I mean "rotation around a local axis." Also, notice that he is using an "AngleAxis" function which should be exactly the same math as I just posted.

10 hours ago, scragglypoo said:

Does it matter what order I put the rotations in? It works either way this*x or this*y first.

Also why does y need to be a world-space rotation? why not x?

In theory, I think if you are doing rotations in local space like this, order shouldn't matter.

You don't NEED to do the Y rotation around the world's up vector, but it helps a lot if you want to maintain an up/down direction as a point of reference for the player (the best example of this is gravity which is in almost every game.) It makes it a lot easier for the player to maintain a sense of orientation.

21 hours ago, missionctrl said:

it looks like changing the order performs the rotation around a locally-centered origin, but not in a local orientation. Maybe that's my fault for mis-using terminology a bit. When I say "local rotation" I mean "rotation around a local axis."

But he does say:

Rotate around a local axis: rotation = rotation * Quaternion.AngleAxis(10, Vector3.Up);
Rotate around a world axis: rotation = Quaternion.AngleAxis(10, Vector3.Up) * rotation;

And it does work for me exactly as your code does (I have to inverse the order however; probably because they're matrices or because I use row order).   But I find your code clearer, and ready to use with an angle.

Edit: I understand now; rotating around local x-axis, and then world y-axis actually mimics real life, if you move your head.