IO completion port

I have some example IOCP code that you could download to play with. It provides some classes which wrap up the complexity somewhat. You can get the code from here: http://www.serverframework.com/products---the-free-framework.html There are also links to the code project articles that I wrote about this code back in 2002.

I have some example IOCP code that you could download to play with. It provides some classes which wrap up the complexity somewhat. You can get the code from here: http://www.serverfra...-framework.html There are also links to the code project articles that I wrote about this code back in 2002.

Thank you very much. I'll take a look at it. Iocp can be a bit confusing when you look at it for the fist time.

Perhaps I'm just lazy. It sounds like you're building something that produces results that are awfully close to what you'd get from a web service like SOAP, so why not just throw a copy of Apache and gSOAP or Axis2 onto your server and be done with it? Most decent RDBMSs include a C/C++ API, so there you go... a fairly secure middle tier up and running in less than a day, and you still get to maintain control at the source code level.

Yes, I know that socket programming can be extremely fun (challenging, rewarding), but I just thought I'd throw the web services route out there. I've done this before with Apache/gSOAP/MySQL Community Edition, and it all worked out extremely well.

P.S. Regarding binary data transfer: "Apache Axis2 supports Base64 encoding, SOAP with Attachments and MTOM (SOAP Message Transmission Optimization Mechanism)."

Thank you very much. I'll take a look at it. Iocp can be a bit confusing when you look at it for the fist time.

Yes, it can, but it's actually fairly straight forward once you get your head around the concept. Remaining async throughout your data processing is the more complex part but that's not too bad if you just build state machines.

The code that I linked to is stable and reliable but it is missing quite a few features that you probably should think about, especially if your server will be sending 'push' notifications to clients that may be on slow network links. I talk about the issues that you can come up against if you ignore TCP flow control with IOCP style async networking APIs here on my blog: http://www.serverframework.com/asynchronousevents/2011/06/tcp-flow-control-and-asynchronous-writes.html

I talk about the issues that you can come up against if you ignore TCP flow control with IOCP style async networking APIs here on my blog: http://www.serverfra...ous-writes.html

Thanks again for all the useful info, it's appreciated.

I made a work for my Operative System class using IOCP for a server, at first it was pretty hard to understand, but when you get the idea of how it works you love it.

To learn a bit more about IOCP I used the book "Windows via C/C++", it has some good pages about it and why it is so good, and of course, an example with comments.

What you said about IOCP is correct, when I did my workd there was no need for a threadpool api, but I'm not really an expert in the windows API, so I can't really help you on that one.

One more thing that's very good about IOCP is that it can manage the number of threads running so you don't have to waste context switches and it tries to maximize the the workd made by the threads, instead of wasting processor time context switching between them.

This is not a direct answer to your question, but as a suggestion you might want to consider looking at Boost.Asio (or non-Boost Asio). It's an asynchronous IO library that basically uses the best available methods on all supported platforms (which includes Solaris). On Windows, it uses IOCP. It includes a high-performance implementation of the highly scalable Proactor framework (and on systems without native asynchronous IO it will emulate it via a Reactor on the backend), and will free you from having to worry about most of the details and gotchas when working with IOCP at a low level.

One more thing that's very good about IOCP is that it can manage the number of threads running so you don't have to waste context switches and it tries to maximize the the workd made by the threads, instead of wasting processor time context switching between them.

Yes, that's one thing i find interesting about it and i actually like the way it works. It should scale very nicely with the number of CPUs (cores) on machines.

we did take a look at SOAP like Taby said but considering the nature of what we're doing, it is not really appropriate. Don't get me wrong, those tools are really neat. I'll take a look at non-boost asio.

I have work on a small prototype and encontered a weird problem with my I/O worker threads. The server thread basically waits on an event that is triggered when a connection is received, her's the code for the thread routine :


int AsyncServer::Run()
{
bool bRunning = true;
DWORD iRet = 0;
HANDLE Events[SE_COUNT] = {0};
if(mIocp.Create(miConcurency))
{
if(mIoWorker.Initialize(&mIocp))
{
Events[SE_ACCEPT_EVENT] = mhAcceptEvent;
Events[SE_SHUTDOWN_EVENT] = mhShutdownEvent;
// Reset the shutdown event.
ResetEvent(mhShutdownEvent);
if(CreateListeningSocket(BACKLOG_COUNT))
{
if(WSAEventSelect(mListener, mhAcceptEvent, FD_ACCEPT) != SOCKET_ERROR)
{
while(bRunning)
{
DWORD iWaitResult = WaitForMultipleObjects(SE_COUNT, Events, FALSE, INFINITE);
switch(iWaitResult)
{
case WAIT_OBJECT_0:
{
ResetEvent(mhAcceptEvent);
//TEST
struct sockaddr_in remote = {0};
int iAddrLen = sizeof(remote);
DESCRIPTOR Socket = WSAAccept(mListener,
reinterpret_cast<struct sockaddr *>(&remote),
&iAddrLen,
NULL,
NULL);
closesocket(Socket);
TRACE(L"\n--- Accepted connection ---\n");
//END TEST
}
break;
case (WAIT_OBJECT_0 + 1):
{
// Shutdown the server.
bRunning = false;
//TEST
TRACE(L"\n--- Server shutting down ---\n\n");
//END TEST
}
break;
}
}
}
}
// Server shutdown.
ShotdownSequence();
}
}
return 0;
}


The mIoWorker.Initialize does this :


bool bRetVal = true;
IOWorker *pWorker = NULL;
unsigned int iThreadCount = pIocp->GetConcurrentThreadCount();

mpIocp = pIocp;


mWorkers.reserve(iThreadCount);

for(unsigned int i = 0; i < iThreadCount; ++i)
{
pWorker = new(std::nothrow) IOWorker(pIocp);
if(pWorker)
{
mWorkers.push_back(pWorker);
bRetVal = bRetVal && pWorker->Start();
}
}
return bRetVal;


So, in order to stop the thread, the mhShutdownEvent event is signaled which breaks the loop and gets to the ShotdownSequence method. The method called to stop the server basically only signal the event then calls WaitForSingleObject on the thread handle.

The ShotdownSequence() method does the following :


void AsyncServer::ShotdownSequence()
{
// Shutdown the I/O thread pool.
mIoWorker.Shutdown();
// Close the iocp.
mIocp.Close();
mbIsRunning = false;
}


and mIoWorker.Shutdown() post a completion packet for each thread and waits for each thread to terminate :


void IOWorkerPool::Shutdown()
{
IOWorker *pWorker = NULL;
std::vector<IOWorker *>::iterator it;

for(unsigned int i = 0; i < mWorkers.size(); ++i)
{
mpIocp->PostIoCompletionOperation(NULL);
}
for(unsigned int i = 0; i < mWorkers.size(); ++i)
{
mWorkers->Wait();
}
while(mWorkers.size())
{
pWorker = (*mWorkers.begin());
delete pWorker;
mWorkers.erase(mWorkers.begin());
}
pWorker = NULL;
}


The problem i have with this is that it freezes once in a while waiting on a thread (ramdom one) when it shutsdown. However, the threads all terminate normaly. If i put a Sleep(2000) as a test instead of the loop that calls mWorkers->Wait(), everything terminate just fine (Sleep is just for a test). The wait method simply calls WaitForSingleObject on the thread handle. I really don't see what happens ....

The I/O workers only do the following so far :


int IOWorker::Run()
{
DWORD iBytes = 0;
ConnectionLink *pLink = NULL;
LPOVERLAPPED pOverlapped = NULL;

while(mbIsRunning)
{
if(mpIocp->GetIoCompletionOperation(&iBytes, &pLink, &pOverlapped))
{
if(pLink)
{
}
else
{
mbIsRunning = false;
}
}
}
return 0;
}


Edit :

Is it wrong to use WaitForSingleObject to wait for a thread to terminate (or at least to call it many times in row) ?

Ok, i found what the problem is and i thought i might as well mention it since it is related to the Win32 thread api. Alot of you probably know that already.

The problem was hiding in my thread class. I was using _beginthread to create the thread. Reading the Microsoft documentation i learnt that this function may not always return a valid handle (especially if the thread terminates too quickly) which can cause WaitForSingleObject to hang. One solution is to call endthread/endthreadex explicitly at the end of the thread function. A better solution is simply to use _beginthreadex which solved the problem. It also uses a thread function that returns a value which will be the return code of the thread (that can be forced with endthreadex).


Edit :

I would also like to add, @krippy2k8, the non-boost asio looks like a very nice library.