GPU VS CPU

Everything that is related to pixel processing must be done on the GPU, there is no other option (unless you want to do software rendering). On the vertex level, you have the choice.

Now, unfortunately, current 3D cards have special video RAM (VRAM), that is onboard of the 3D card, and not connected to the main memory bus. That VRAM is connected with the CPU by the AGP bus, and this is almost always a bottle neck. Esp. readin back the data to CPU-accessible main memory is extremely slow. The ultimate solution would be high speed unified memory, in the normal address space, both accessible from CPU and GPU. This is undoubtly where the technical evolution will go, but currently, we have to make the best out of what we have. Best means maximum performance.

To get best performance, the data required to render a scene (vertices, faces, textures, shaders, etc) must be easily accessible by the GPU. Let''s take vertex data, for example, as typically lots of operations are performed on it. If you keep your vertex data in host memory (your normal RAM), then you will need to transfer it to the 3D card every frame, squeezing it through the slow AGP bus (CPU needs to copy to AGP memory, GPU needs to pull via DMA -> slow) A better solution is to manually copy the vertex data to either AGP memory (where the GPU can directly access it), or the VRAM, and keep it there. But now the CPU can not access it anymore (at least not in an optimal way: VRAM has the AGP bus in the way, AGP memory is uncached).

So basically, if you want your data to stay in VRAM, you will have to do all processing on the GPU, since it is the only processor that has fast access to that memory. But a GPU is no general purpose processor, it is highly specialized. In other words, you have lots of limitations on what you can and ccannot do with your data. If we stay with the vertex data example, you can process it by using vertex shaders. Those are small programs running on the GPU.

But they impose an important number of limitations, the most important ones being:
A) you can only access a single vertex at a time
B) you cannot create a new vertex
C) you cannot destroy a vertex

It''s a tradeoff: flexibility versus performance. If your processing fits within the limits of the GPU, and is not too complex, then it should be done on the GPU (for example, matrix transforms, basic lighting, skinned animation, etc). In all other cases, do it on the CPU (eg. complex physics, complex lighting models, etc).

Now, note that the GPU is not always faster than the CPU at performing those tasks, esp. if using 3DNow and friends. But it is the only way to keep your data in high speed memory, and avoid unneeded slow data movement.

Another important point to keep in mind, is that CPU and GPU are two separate and independent units. They can execute code at the same time. If you have a programmable 3D card, then you basically have a multiprocessor system. Taking advantage of that fact is very important to get really high performance.