Where to next?

Animation, then maybe some physics and collision, to turn it more towards a game environment. A math library for starters (Quats, vectors, matrices, ...).

Also, it looks like it could use some cool dynamic lighting.

Once you get more interactions, there will be thinks you will want to add, like particle system, sound, AI, a game framework, ect...

I'd really like to get this camera to work the way I'd like it to, are quaternions really hard for someone who is just now learning Calculus?

Rotation matrices are easier to visualise, since the column / rows represent the axis of orientation. Quaternions are more mind bending. In the context of game maths, It's hard to get a visual representation of them, but it's easier if you consider them as some sort of axis-angle representation of a rotation.

Quaternion(x, y, z, w).

(x, y, z) -> axis of rotation.
w -> cos(angle of rotation / 2).

I use them without going into too much detail. In their simplest form and usage, they are not too bad. Use them as a tool for basic maths routines, quaternion<->euler<->matrix<->vector operations and conversions, that should be enough. They are great for cameras, and skeleton frames for animations.

Once you have that math library ready, it will make your life a whole lot easier when you start doing physics, animation, collision and AI. It's good to have the choice of using quaternions vs matrices.