Streaming geometry

I've done some work with similar stuff:

You want to divide the "world" up into some logical units (for a terrain engine, something like 1km*1km blocks), and then use a predication algorithm to work out two things:
1. Which are required (as in, those that are on screen now, or likely to be if the user suddenly turns around)
2. Which are likely to be needed soon (e.g. those "forwards" in the direction the user is walking/driving)

You then want to work on some sort of "Working Set" space, where you have a given pool of these sections (tip: making it scaleable works on low end but is also free optimization for the high end) working as a Least Recently Used (LRU) cache. When the working set is full, you relegate the least recently used section back to system memory (and/or purge it complety) and replace it with something more relevent.

As for actually loading in the data, try and store a low detail model as a placeholder - whereby the user can still see something on screen if they "overtake" your own manager. This is noticeable in GTA:Vice-City, get a fast bike and nip from one end to the other really quickly and you'll see most of the buildings are just cubes with blurry low-res textures. After a couple of seconds it catches up and you see the full detail/proper representation...

Streaming the data in as a given time slice per frame (say 5ms per frame) and pausing only if the full dataset is required but not in memory. For optimal performance with a graphics API I recommend streaming it into a SYSMEM array/structure before copying it to VRAM - constantly locking/writing/unlocking small bits of data to a Vertex Buffer each frame hurts.

Someone, in a previous thread like this, pointed out that multithreading can be really good - as a lot of physical-disk read operations can result in IO waits. I haven't tried this optimization though so can't vouch for it - but it does make sense.

hth - typed all that really quickly, so might've missed something out. Let me know if you want to know more about the specific parts. I worked this stuff out myself so don't have any nice web links/articles to point you towards!

Cheers,
Jack