Skip to main content
gamedev.net
🔒 Locked

Streaming resource loading / Multithreading

Started by janta May 29, 2006 at 11:36 PM 0 replies 950+ views
Original Post
janta
janta
Hello, I'm glad the forum is back to life :) Before I start designing sth, I'd like to request advices on the subject of streaming resource loading during a game. Here is the -common I think- design I've thought of (with pieces borrowed here and there as some people may notice) I plan to have a resource loader which runs on it own thread. That thread would be sleeping until one sends him a request to load some file. That request would have the form of stacking a resource identifier (most likely an integer) on the resource manager "loading command stack". It would go back to sleep when it's stack is empty, and also, it would know how to resolve to the real data file that the ID refers to. Since I dont want the caller to wait for the resource to be loaded (that could take a couple of seconds, even more), I thought that I could use an asynchronous pattern. What I plan to do is pass a pointer to an Interface along with the resource identifier. Somthing like:
void ResourceManager::LoadResource(UINT _uiResID, ResourceBase* _pCaller);

class ResourceBase
{
    virtual void SendMessage(int _msg);
}     
Calling this method would have the effect of waking up the loader thread and have it load the resource and fill appropriate fields of _pCaller in a thread safe fashion. When loading is complete, the thread would call:
_pCaller->SendResource(RESOURCE_READY);
to notify it that it can start using its resource. So what is your opinion about this first point ? Second point: I want to ensure that loading the resource is "load balanced". What I mean is that there is no point is having a separate thread loading a resource if that thread holds CPU monitoring (assuming I'm running on a single core) during all the loading process. So lets say I'd use the antique yet powerful fread() to read the data within some big file (like 20 MB) Is there any chance that a call like
fread(pData, 20 * 1024 * 1024, 1, pFile);
would yield the cpu's attention to another process ? If no, then it is pointless. And would that code be better (less efficient, but pure speed doesnt matter I just want it to be treated in harmony with the other threads:
for(int iChunk = 0 ; iChunk < iNbChunkInFile ; iChunk++)
{
    fread(&pData[iChunk*sizeOfChunk], sizeOfChunk, 1, pFile);
}
Or do you have any better suggestion ? As you can see I'm no expert of multithreading, yet I believe that it is the future and I want to learn :) Thanks for your reading ! Regards, Janta
jkleinecke
jkleinecke
You can set your background resource thread to a lower priority than your main one. By default the OS will give each thread a time slice to execute in. You need to have the background thread yield to the main thread when its not doing any work.

The only time you'll be loading in a large a amount of data is during initialization. During the game, your resource streaming should be relatively small and shouldn't be too much of a problem. I think that the OS will suspend the background thread until the hard drive can fetch the data, but I could be wrong.
-----------------------------------Indium Studios, Inc.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.