Skip to main content
GameDev.net gamedev.net
🔒 Locked

Partial Thread Synchronisation

Started by Chicktopus Apr 19, 2011 at 9:05 PM 18 replies 3k views
Original Post
Chicktopus
Chicktopus
Ahoy shipmates!

I'm using the Win32 API for this 'un.

Basically what I have is a main thread which creates a child thread. Now, the child thread won't get it's parameters because the creating function returns before the thread gets started. Normally I'd use something like WaitForSingleObject() to get around this, but the child does a few assignments and enters an essentially infinite loop, so what I need is for the main thread to continue on about it's jolly business after the assignments have been done at the start of the child process. I've tried using events and mutex's, but I can't get 'em working. I'll post what I've got:



//HEADER
HANDLE g_assignEvt;


static DWORD ThreadFunc(void *p_param)
{
((Collision*)p_param)->ApplyEffect(p_param);
return 1; // Return error
}





//MAIN THREAD


HANDLE Collision::Start(ColParams p_params)
{
g_assignEvt = CreateEvent(NULL,TRUE,0,TEXT("AssignEvt"));
DWORD m_threadID = 0;
LPVOID* m_argList = (LPVOID*)&p_params;
g_thread = CreateThread(NULL,0,(LPTHREAD_START_ROUTINE)ThreadFunc,m_argList,0,&m_threadID);

WaitForSingleObject(g_assignEvt, INFINITE); //Main thread can't seem to detect that the child has triggered the event, so this just waits forever

CloseHandle(g_assignEvt);
return g_thread;
}//Start



//CHILDTHREAD


void Collision::ApplyEffect(void* p_params)
{
ColParams* m_temp = (ColParams*)p_params;
ColParams m_params = *m_temp;
const unsigned int p_planeCount = *m_params.planeCount;
const unsigned int p_sphereCount = *m_params.sphereCount;
Plane* p_plane = m_params.planes;
Sphere* p_sphereArr = m_params.spheres;
float p_time = *m_params.time;
bool* m_terminate = m_params.terminate;

SetEvent(g_assignEvt); //Assignments are done

...

while(!g_terminated)
{
...
}
}






Obviously I've cut out anything irrelevant. I've tried using OpenEvent(), but this succeeds when I try something like OpenEvent(EVENT_MODIFY_STATE, TRUE, TEXT("AssignEvt")) or (OpenEvent(READ_CONTROL, FALSE, TEXT("AssignEvt")) but fails with error code 5 'Access denied' when I try (OpenEvent(PROCESS_ALL_ACCESS, FALSE, TEXT("AssignEvt")). I'm not sure I ultimately even need to use OpenEvent, that's why I'm here. I've also tried DuplicateHandle() with little success. I may just be doing something wrong.

In any case, even if SetEvent() is hit, it doesn't seem to make a difference to the WaitForSingleObject() in the main thread. Posiible it's using different handles? I've tried passing them through to the child thread too, but that didn't work. Any help would be greatly appreciated.

Oh, by the way, I'm not supposed to be using any special libraries or anything for this project.
Red Ant
Red Ant

Ahoy shipmates!

I'm using the Win32 API for this 'un.

Basically what I have is a main thread which creates a child thread. Now, the child thread won't get it's parameters because the creating function returns before the thread gets started.


Ok, stop right there. You are going about this completely the wrong way. I understand what you're trying to do (making your Collision::Start() method stick around long enough for your newly started thread to make a copy of the data to which it has been given a pointer), but it's not a good idea. Instead of putting that data on the stack in your Collision::Start() method, what you __should__ be doing is to create the data using operator new and then just transferring ownership of that heap-allocated data to your thread, which will of course then be responsible for deleting it when it's done with it.
Red Ant
Red Ant

Oh, by the way, I'm not supposed to be using any special libraries or anything for this project.


Too bad, I was just going to suggest using boost::thread. The bare WIN32 threading API is a horrible environment to write a multi-threaded application in. Just way too many ways of shooting yourself in the foot.


EDIT: By the way, you can't (reliably) have threads communicate with each other like that:


while ( !g_terminated )
{
}


Any and all concurrent access to shared data must use some form of sychronization, or else the results are undefined. Each time you want to inspect or alter the value of your g_terminated variable, you must first acquire some kind of lock (read up on critical sections ... use the msdn online documentation). Otherwise all kinds of weird things can happen, such as one thread changing the value of g_terminated but the other thread never seeing the change.
Chicktopus
Chicktopus

[quote name='Chicktopus' timestamp='1303247147' post='4800511']
Oh, by the way, I'm not supposed to be using any special libraries or anything for this project.


Too bad, I was just going to suggest using boost::thread. The bare WIN32 threading API is a horrible environment to write a multi-threaded application in. Just way too many ways of shooting yourself in the foot.


EDIT: By the way, you can't (reliably) have threads communicate with each other like that:


while ( !g_terminated )
{
}


Any and all concurrent access to shared data must use some form of sychronization, or else the results are undefined. Each time you want to inspect or alter the value of your g_terminated variable, you must first acquire some kind of lock (read up on critical sections ... use the msdn online documentation). Otherwise all kinds of weird things can happen, such as one thread changing the value of g_terminated but the other thread never seeing the change.
[/quote]

Hah! yeah. I was pawing at boost earlier. Oh well!

Well, I've tried initialising each variable using the the new operator. So far so good. I'm having a little trouble deleting them, though. Getting a few Debug Assertions. I can probably fix that by myself, though. One thing before I continue any further: The data I'm passing in to this thread is all constantly being updated by the main thread. I need these changes to change the data in this child thread, too. Is that going to happen if I use new? I guess the data in the main thread should update the exact same variables I'm passing to the thread, so I'm updating the data at the address, which means I then just worry about deleting them all in the main thread too?
Red Ant
Red Ant

Well, I've tried initialising each variable using the the new operator. So far so good. I'm having a little trouble deleting them, though. Getting a few Debug Assertions. I can probably fix that by myself, though.


Sounds like some sort of heap corruption. Could you be trying to delete the same thing more than once?


One thing before I continue any further: The data I'm passing in to this thread is all constantly being updated by the main thread. I need these changes to change the data in this child thread, too. Is that going to happen if I use new? I guess the data in the main thread should update the exact same variables I'm passing to the thread, so I'm updating the data at the address, which means I then just worry about deleting them all in the main thread too?


Well, if you need access to that data both from within your main thead and from within your worker thread then you need to give them both a reference or a pointer to that data. If the data was allocated via new then of course you must take care to only delete it once. It's up to you to work out who should assume ownership (and thus responsibility to clean up) of the data. Also, remember that you will need to make a CRITICAL_SECTION and use EnterCriticalSection() / LeaveCriticalSection() to synchronize access to that shared data.

EDIT: Just to make sure no misunderstandings arise from this, I wasn't suggesting that you allocate each and every variable via operator new, just that ColParams struct that you want to pass to your child thread.


// Change this ...
LPVOID* m_argList = (LPVOID*)&p_params;
g_thread = CreateThread(NULL,0,(LPTHREAD_START_ROUTINE)ThreadFunc,m_argList,0,&m_threadID);

// to this. This uses operator new and the copy constructor of class ColParams to make a heap-allocated copy of the parameters you're passing. Btw, since you've said that this data should be continuously updated by your main thread, you will also have to store a pointer to this instance somewhere your main thread can get at it from.
ColParams* const copyOfParams = new ColParams( p_params );
g_thread = CreateThread( NULL, 0, ThreadFunc, copyOfParams, 0, &m_threadID );


Chicktopus
Chicktopus
Hmm. Yeah. I have all the data transferring over ok, but it's not being updated in the child thread as it is in the main thread. Weird thing is that the value won't change once the function assigning the parameters to the struct returns, in this case the Window::OnCreate(). The child thread will update values after the thread has started but before the function assigning the parameters returns. I'd say it's a scoping issue, but the parameters struct and all it's variables are global.

So basically, I have:



//Collision.h

struct ColParams
{
unsigned int* sphereCount;
unsigned int* planeCount;
float* time;
Sphere* spheres;
Plane* planes;
};






//Window.h

#include "Collision.h"


ColParams g_colParams
//Window.cpp
void Window::OnCreate()
{
...
g_renderCount = 0;
g_planeCount = g_cube.GetPlaneCount();
g_time = 0;
g_sphereArr = NULL;
g_planes = g_cube.GetPlanes();


//Assign parameters to the struct

g_colParams = new ColParams();
g_colParams->sphereCount = &g_renderCount;
g_colParams->planeCount = &g_planeCount;
g_colParams->planes = g_planes;
g_colParams->spheres = g_sphereArr;
g_colParams->time = &g_time;

g_collision.Start(g_colParams);
//<----------- if I update here, it's change it in the child thread
}//OnCreate
void Window::OnDisplay()
{
planeCount = 200; //<---------- Yet if I update it say here, it'll only change in this main thread
}//OnDisplay






//Collision.cpp



void Collision::ApplyEffect(void* p_params)
{
ColParams* m_temp = (ColParams*)p_params;
ColParams m_params = *m_temp;

const unsigned int p_planeCount = *m_temp->planeCount;
const unsigned int p_sphereCount = *m_temp->sphereCount;
float p_time = *m_temp->time;
Plane* p_plane = m_temp->planes;
Sphere* p_sphereArr = m_temp->spheres;
delete m_temp;

...

while(1) //Just run forever for now, until I figure out a better way to stop the thread
{
m_planeCount = m_planeCount; //Check to see if anything's been updated
...
}
}//ApplyEffect


*Edit* Just made a little clarification
the_edd
the_edd
EDIT: I was battling with the layout of this post for a while. My apologies for it all being crammed together, seems like the forum is eating my line breaks.



ColParams g_colParams
//Window.cpp
void Window::OnCreate()
{
// ...
g_collision.Start(g_colParams);
//<----------- if I update here, it's change it in the child thread
}//OnCreate
void Window::OnDisplay()
{
planeCount = 200; //<---------- Yet if I update it say here, it'll only change in this main thread
}//OnDisplay



Here you are updating a local variable. What does this have to do with g_collision?

But it's *really important* that you pay close attention to what Red Ant said about synchronization, particularly the use of CRITICAL_SECTIONS (which the rest of the world outside of Windows calls a "mutex"). You simply cannot expect anything to work unless a mutex (or another synchronization mechanism) is used.



void Collision::ApplyEffect(void* p_params)
{
ColParams* m_temp = (ColParams*)p_params;
ColParams m_params = *m_temp;

const unsigned int p_planeCount = *m_temp->planeCount;
const unsigned int p_sphereCount = *m_temp->sphereCount;
float p_time = *m_temp->time;
Plane* p_plane = m_temp->planes;
Sphere* p_sphereArr = m_temp->spheres;
delete m_temp;

...

while(1) //Just run forever for now, until I figure out a better way to stop the thread
{
m_planeCount = m_planeCount; //Check to see if anything's been updated
...
}
}//ApplyEffect

[/quote]

What is this while loop supposed to be doing? It's not checking anything. It's also not clear to me which thread is supposed to be running which part of your code.

If I were you, I'd forget about Windows and Collision systems and concentrate on getting the basics right w.r.t to synchronization. Once that's nailed, move back to your real code.

Wrapping up Red Ant's suggestion in to something reusable, we have:


// Uncompiled/untested

#include <windows.h>
#include <process.h>

#include <cassert>
#include <memory>

namespace detail
{
// implementation detail of create_thread
template<typename Callable>
unsigned thread_body(void *callable)
{
assert(callable);
std::auto_ptr<Callable> functoid(static_cast<Callable*>(callable));
(*functoid)();
return 0;
}
}

// Create a thread (whose HANDLE is returned), that runs a *copy* of functoid,
// which should be a callable object or pointer-to-function.
// Use join_thread() to wait for termination.
template<typename Callable>
HANDLE create_thread(const Callable &functoid)
{
Callable *copy = new Callable(functoid);
uintptr_t h = _beginthreadex(0, 0, &detail::thread_body<Callable>, copy, 0, 0);

if (h == 0)
{
delete copy;
return 0;
}

return reinterpret_cast<HANDLE>(h);
}

// Wait for the specified thread to finish.
// It is assumed the thread was created with create_thread.
bool join_thread(HANDLE thread)
{
assert(thread);
bool ok = WaitForSingleObject(thread, INFINITE) == WAIT_OBJECT_0;
if (ok) CloseHandle(thread);
return ok;
}


Given that, we can do things like


struct MyThread
{
MyThread(int *a, const std::string &b) : a(a), b(b) { }
void operator() () { /* thread body goes here, can use 'a' and 'b' */

int *a;
std::string b;
};

int main()
{
// The thread we create here gets its own copy of the MyThread object
// we create, so there aren't any races. However, given that MyThread::a is a pointer,
// the thread we create now effectively has access to our local variable 'a':
int a = 42;
HANDLE h = create_thread(MyThread(&a, "hello"));


if (!h) return 1;
// ...

join_thread(h);
return 0;
}
Chicktopus
Chicktopus
Well basically the idea was to have a thread that tests for collisions between objects in one thread and updates them in another. I've actually managed to get that working now and the variables are being updated fine, except for my sphere array. The different thing about the sphere array is that on every frame it is deleted and re-created. This is necessary because the spheres are retrieved from a server, updated and then the results sent back. The same spheres aren't necessarily going to be used on the next frame. Basically what I have is:


Initialise a sphere array of 0 elements

Start a thread checking for sphere collisions and writing to the sphere array if there are spheres present

while(true)
{
Get the number of spheres from the server
Create an array of spheres within a range of the mouse position using the new operator

If there are any spheres present,
{
delete the array
}
for( i = 0; < number of spheres; i++)
{
Get each sphere data component from the server and assign to a temp vector.at{i)
}
for (k = 0; k < number of spheres; k++)
{
Update these sphere
Draw the sphere
Send the new values back to the server
}

}


Problem is that the sphere array always seems to be unassigned. possibly because I create it in a different method to that which I start the thread? Would using a vector maybe combat this?
Chicktopus
Chicktopus

What you're describing is a good candidate for a task-driven approach, rather than creating a fixed number of threads for specific things. Have you explored things like Microsoft's Parallel Patterns Library?
 

 Alas, I'm not allowed to use any special libraries like that, I've got to use the old fashioned methods.

I've actually got the thread working ok (I think). I've created another thread which sends/receivesdata from the server and updates the spheres, which runs parallel to the collision detection thread. Now the main thread is supposed to be drawing while this second thread updates. I'm getting some memory access violations on the second or third iteration though, so I'm trying to get that one sorted
rneckelmann
rneckelmann

EDIT: By the way, you can't (reliably) have threads communicate with each other like that:


while ( !g_terminated )
{
}


Any and all concurrent access to shared data must use some form of sychronization, or else the results are undefined. Each time you want to inspect or alter the value of your g_terminated variable, you must first acquire some kind of lock (read up on critical sections ... use the msdn online documentation). Otherwise all kinds of weird things can happen, such as one thread changing the value of g_terminated but the other thread never seeing the change.


In that specific case, what could go wrong?

If the child thread is simply checking whether g_terminated has been set to true, I can't see how it could cause synchronization problems. If someone was reading the value, doing something with it, and then setting another value - then we'd have a synchronization problem.

From Interlocked Variable Access (Windows) on MSDN:

Simple reads and writes to properly-aligned 32-bit variables are atomic operations. In other words, you will not end up with only one portion of the variable updated; all bits are updated in an atomic fashion. However, access is not guaranteed to be synchronized. If two threads are reading and writing from the same variable, you cannot determine if one thread will perform its read operation before the other performs its write operation.
[/quote]

Busy-polling a variable might be a bad idea for other reasons, like hogging up excessive CPU resources.
Red Ant
Red Ant

[quote name='Red Ant' timestamp='1303248265' post='4800520']
EDIT: By the way, you can't (reliably) have threads communicate with each other like that:


while ( !g_terminated )
{
}


Any and all concurrent access to shared data must use some form of sychronization, or else the results are undefined. Each time you want to inspect or alter the value of your g_terminated variable, you must first acquire some kind of lock (read up on critical sections ... use the msdn online documentation). Otherwise all kinds of weird things can happen, such as one thread changing the value of g_terminated but the other thread never seeing the change.


In that specific case, what could go wrong?

If the child thread is simply checking whether g_terminated has been set to true, I can't see how it could cause synchronization problems. If someone was reading the value, doing something with it, and then setting another value - then we'd have a synchronization problem.
[/quote]

Are you aware of the memory visibility issues that doing so could entail? For instance, the OS might decide that thread A should move g_terminated to some kind of cache and then access that for increased speed. Meanwhile, thread B (whose version of g_terminated resides in RAM) modifies g_terminated. However, it is not aware that thread A has moved its version to a cache, so thread A never gets to see the changed value of g_terminated.
Chicktopus
Chicktopus
Ok, I do believe I have fixed it all. Critical sections were't working for me, but mutexes worked like a charm. Problem resolved!
Red Ant
Red Ant

Ok, I do believe I have fixed it all. Critical sections were't working for me, but mutexes worked like a charm. Problem resolved!


Beware that compared to CRITICAL_SECTIONS, mutexes (or what Windows calls mutexes anyway) are relatively heavy-weight. At any rate, there is no reason why critical sections shouldn't "work for you". Can you elaborate on what kind of problems you've run in with them?
rneckelmann
rneckelmann

Are you aware of the memory visibility issues that doing so could entail? For instance, the OS might decide that thread A should move g_terminated to some kind of cache and then access that for increased speed. Meanwhile, thread B (whose version of g_terminated resides in RAM) modifies g_terminated. However, it is not aware that thread A has moved its version to a cache, so thread A never gets to see the changed value of g_terminated.


I'm pretty sure cache coherency is ensured on all modern multicore system?

It's not guaranteed which order the caches will be notified if you change a variable like in the above example, but they'll all know it eventually. If thread A is just waiting for thread B to set a bool to true, it will get the new value eventually when the cache is updated.

Some quick googling led to this: http://en.wikipedia.org/wiki/MESI_protocol
Hodgman
Hodgman

It's not guaranteed which order the caches will be notified if you change a variable like in the above example, but they'll all know it eventually. If thread A is just waiting for thread B to set a bool to true, it will get the new value eventually when the cache is updated.
In the specific case of [font="CourierNew, monospace"]g_terminated[/font], caching is probably not an issue (on any multi-core architecture I've used), but in other cases it's a crash waiting to happen, e.g.//thread A
g_pData = new Data();

//thread B
if( g_pData )
g_pData->DoStuff()
In the above case, the pointer value could become visible to therad B's caches before the actual values that are pointed to by the pointer.
i.e. the pointer assignment could appear to happen before the constructor!
It's good practice to always ensure proper thread synchronisation/ordering, unless you've got a very good reason to risk dealing with out of order memory issues.

However, going back to [font="CourierNew, monospace"]while( !g_terminated )[/font], if [font="CourierNew, monospace"]g_terminated[/font] isn't volatile, then the compiler can rightfully decide to optimise that code to fetch the value only once://pseudo machine code
bool fetch = ReadRam(g_terminated);
while( !fetch )
instead of what you want - fetching it each iteration:bool fetch;
while( !(fetch = ReadRam(g_terminated)) )
which would cause an infinite loop (but because of bad assembly, not caches )
Again, best to just do the right thing and wrap your shared data up in a critical-section/mutex to avoid these problems (unless you've got a good reason not to).

N.B. A critical-section / mutex will have the same effect on the compiler as volatile will - it will stop the optimizer from moving the read instruction across the critical section's boundary -- if you're doing things at a high level and by the book, you shouldn't ever have to use the volatile keyword yourself.
rneckelmann
rneckelmann

Again, best to just do the right thing and wrap your shared data up in a critical-section/mutex to avoid these problems (unless you've got a good reason not to).


Yes yes, I agree with you 99%. My only caveat is that critical sections and mutexes can be quite heavy if overused. Atomic variables with InterlockedExchange()/whatever are often more efficient. Also, the best way to deal with synchronization is to avoid having to do it

Good point about remembering the volatile keyword when needed, that fixes one potential bug in the code base I'm working on now
Kaptein
Kaptein
if you can't afford wait times (mutexes and timers cause them), you can use round-robins so that only one thread is doing work on something at a time

thread a:

if (a == 0)
do work
a = 1;


thread b:

if (a == 1)
do work unless too much time is spent
if (the work is completely done) a == 0;
the_edd
the_edd

if you can't afford wait times (mutexes and timers cause them), you can use round-robins so that only one thread is doing work on something at a time

thread a:

if (a == 0)
do work
a = 1;


thread b:

if (a == 1)
do work unless too much time is spent
if (the work is completely done) a == 0;


This will only work sporadically unless memory barriers are employed.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.