To be good, you can't know enough math.
Saying a computer programmer, especially a 3D game developer, doesn't really need to know much math is like saying a professional musician doesn't need to know anything about scales or rhythm or harmony or composition; all they need to do is be able to play like the guy in the You-tube video.
You need to know algebra: linear algebra (vectors, matrices, transforms and transformations, convolutions), set theory (groups, rings, orderings), combinatorics (permutations, combinations), graph theory, number theory (basis, generators, PDFs/CDFs/PRNGs, series and sequences). Error minimization when approximating real-valued functions using floating point or fixed-point representation. Computability theory and algorithmic analysis. Basic differential and integral calculus of arbitrary dimension is a prerequisite for an understanding of most physics.
If you don't like math, you may just be in the wrong trade. Now, go and practice your scales.