To control threads?

The more threads per core the lower the performance. Context switches are very expensive. Generally, the approach to systems like PS3 or 360 is to aim for 1 thread per core, with potentially a couple running on the main core (a main thread and a worker thread).

Another challenge with threaded systems is lock/resource contention. Putting 2 objects on separate threads won't necessarily gains you anything performance-wise if they require read/write access to the same memory resource to update successfully. For instance an AI entity will need to know where the player is, but if the player is updating on a separate thread you may have contention over accessing the player's position from the AI entity's thread because the player has that memory locked for update.

Yet another is that in games, there are tons of serially dependent update sequences. For the AI to update it may need to know the results of the physics update or the player input update. So if you have AI and physics in separate threads then you run the risk of updating the AI with last frame's physics results for some objects, and this frame's physics results for others. So depending on the needs of various systems you cannot necessarily parralelize those interdependent sequences.

A common solution to many-core development is a job-based system where you have n worker threads updating a load-balanced set of discreet jobs. This will maximize core usage, but because of the above challenges presents a very difficult problem to solve if you want to apply this to games. To my knowledge no one has found a great solution to this for games, but current attempts generally involve double-buffered shared memory spaces (only updater writes, anyone can read) and a tolerance of operating on out of date information. You will also need to implement a set of barriers so that threads across all relevant cores wait to proceed to step 2 until all threads have completed step 1 (basically managing serial dependencies while trying to do things in parallel that can do so).

It's a giant wasps' nest of hassle. Enjoy the pursuit, it's a fun problem to try and crack.

-me

I think I will go ahead and create my thread manager system. At worse performance I assume it would run as fast as a single threaded system right?

I mean, because if every thread is waiting on the currently running thread it only can slow down so much if created and managed correctly.

My manager would initialize based on how many cores the computer is running and swap componants to different threads based on thread performance to keep it as efficiant as possible at any given time.

You are right though, a truely advanced physics system would have to rely on AI and visa~versa. The best performance boosts would be with rendering and things that only need to see values like HUD systems.

I think we need a revolutionary memory design to take full advantage of multi core systems. Something along the lines of allowing multiple cores to access and change the data at the same time or something... but anyway, thank you for the responce.

-TonyG

I am not exactly sure what you are getting at. For simple job dispatching systems you can use a threadpool system that will automatically run the thread on an unused or underused core. Otherwise if you are creating long running threads, you generally want to dedicate them to a core and not swap them around.

maybe I am going about this wrong but my manager takes in account three types of entitys:

Componant obj (contains stuff for the componant like rendering stuff)
Thread obj (contains vector of componant objs and a system thread)
Core

A componant can run on any thread, with multiple componants on any given thread.
Thread objects are created based on number of cores at startup.

So lets say I have 2 cores therefore 2 threads and I have also 4 componants that are completely independant of each other. The componants would have a value based on how long it takes to run their updates.

the idea would be to evenly distrubute the load like:
thread 1 (comp1: 40%, comp2: 10%)
thread 2 (comp3: 30%, comp4: 20%)

My question was.. would this be faster than just making 4 threads and let the OS take care of it because that would increase context switching but reduce my thread management cost.

If you are going to have more threads than cores, it might be in your best interest to let the OS handle it, however, the best practice would be to only create as many threads as cores. Your method only introduces extra overhead.

However, if your threads are following the pattern work->wait->work->wait->work, you might want to reconsider your design and use background workers or some other threadpool implementation. Otherwise, you are simply going to waste time as you are not guaranteed to get any performance by offloading the thread just to collide with the work portion of another thread.

Ideally, if you are going to create a full thread that runs indefinitely, you want it to be something that is going to keep the core busy. As I said, if you are just queing up little jobs, you should look into a threadpool/background worker implementation.

Just looked up thread pool. That seems to be what im doing with the exception that I wasnt designing for "tasks" as much as full componants. That might work alot better but will require a redesign of my componants. Thanks for the replies

Quote:

Original post by agaudreau Just looked up thread pool. That seems to be what im doing with the exception that I wasnt designing for "tasks" as much as full componants. That might work alot better but will require a redesign of my componants. Thanks for the replies

If you can break the components into tasks (and this can be often done although it would not seem like it at first), that will allow you to take advantage of the OS's scheduling to reduce collisions and context switching. I am not sure about other implementations, but the .NET threadpool will keep contexts open so a task can slip right in and do its work, without the overhead of creating a new thread context everytime you need work done.

Quote:

Original post by agaudreau You are right though, a truely advanced physics system would have to rely on AI and visa~versa. The best performance boosts would be with rendering and things that only need to see values like HUD systems.

Actually, rendering is the one place where you get no benefit from multiple threads. The GPU is already maximally paralelized and only a single thread can access the device at a time.

Quote:

Original post by agaudreau I think we need a revolutionary memory design to take full advantage of multi core systems. Something along the lines of allowing multiple cores to access and change the data at the same time or something... but anyway, thank you for the responce.

No, we already have that. That's how shared memory multi-processing works. All threads can concurrently access the memory and that's the problem algorithmically. If two threads are trying to write data to the same class or set of variables they will do it and you'll end up with garbage (some of thread A's bits, some of thread B's. What you aim for with multi-threading is to allow simultaneous reading of memory but prevent simultaneous writing and prevent writing while also reading. And you need to do that while cutting down on lock contention and compare-and-swap operations which get expensive.

But otherwise, what I'd suggest to you after following this thread is for you to take a month or more to research already existing solutions to multi-threaded problems. This isn't even a remotely new field. People have been writing distributed apps for decades, just not games. So study up on the body of algorithms that exist and figure out ways to apply them to your game.

-me

Yeah I didn't mean have more than one thread rendering, I was saying the over all system gains from it because one thread can be rendering while another updates sounds or something.

I realize I am not the first to look at this and I also understand that it isnt as easy as it looks at first. I was just wondering about my manager idea.

And also, I meant a new memory system that would bypass the issues when writing and reading at the same time. just more of a musing that anything serious.

Quote:

Original post by agaudreau thread 1 (comp1: 40%, comp2: 10%) thread 2 (comp3: 30%, comp4: 20%) My question was.. would this be faster than just making 4 threads and let the OS take care of it because that would increase context switching but reduce my thread management cost.

That's not optimal way to distribute work. Percentages also don't mean much with respect to this.

Without further specifying what "component" is, there is a fairly optimal way to use worker threads and simple queuing to do what you want.

I'll use term service instead of component. First, let each service have a update() method. Next, each worker thread looks like this:

while (running) {  Service s = scheduled.popFirst();  if (s != null) {    s->update();    scheduled.pushBack(s);  } else {    // wait till there's work to be done  }};

where scheduled is a FIFO queue.

You then spawn one worker thread per core, and add as many services as needed. As always, the devil is in detail, and it's the "wait till there's work" that can be tricky to implement efficiently.

Scheduler itself can be implemented in a lock-less manner, and above achieves 100% CPU utilization. It's as efficient as it gets for a generic problem.

Of course, the above problem is somewhat useless for real-time simulation where you end up with fixed time step, the need to wait for user input and heavy sequential dependencies between individual services. While the concept remains the same, the implementation needs to be adjusted somewhat to account for those factors.

See overview of Intel's tech demo for overview of issues involved and some proposed solutions.