A Win32 Approach to Multi-threaded Servers Part I - WinSock
by Kurifu Roushu

Thanks to Smoogle for editing this article for me.

Preface

This article I intend to be the first of a few articles, to form a series, in which I will use to take a pretty much ground up approach to creating a multi-threading game server using Win32. Many of you may have seen me poking around the forums in GameDev asking a few questions so that I may more readily complete this article with as much information as possible.

Within this series I do assume that you have a basic understanding of Win32. Some understanding of WinSock would be beneficial, however it should not be required since this is what the first part of the article series will be covering.

The WinSock code in this article will be written as close to the BSD standards as I can make it, so that theoretically the code could be easily ported over to run on any Berkerly Socket Descriptor aware operating system, such as Linux, FreeBSD, and so forth.

The small application source code included (main.c) was originally written to be compiled and use in a Linux environment with gcc. This is the machine I had at hand when I wrote this code, and should also provide a basic overview of the differences between Win32 and Linux based socket code.

Introduction

One of the first things that you need to know about Windows Sockets is that there are three types of sockets that exist: blocking, non-blocking, and asynchronous. Though asynchronous and non-blocking are sometimes seen as very similar, they are not, and shortly you will learn why.

Blocking sockets are what I would call your regular everyday plain old sockets. They hold one connect (just like the others), and when you make a call to recv(), send(), or accept() they will stop the program execution and not return until in incoming connection is made, or data is sent or received. These sockets are the basis to a multi-threading server and will be what we use from this point forth in the documentation.

Non-blocking sockets work in the same manner that blocking sockets do with one exception. Function such as recv(), send(), and accept() will return even if there is no information waiting and program execution will continue normally. The problem that these sockets present is that because they may not return data, you have to keep watching them using very tight loops to actually get the data. These loops may consume unneeded CPU cycles in your application, especially if the code is not optimized correctly.

And last but not least, Asynchronous sockets, which may be sometimes mistaken as non-blocking socket - though they are not - use the Windows Messaging queue to notify the application when it is ok to send, when there is an incoming connection, and when there is incoming data. Note though that the accept(), recv(), and send() functions are in fact blocking functions and will not return until properly executed, however since Windows should not be notifying us unless data is actually present, this does not present a problem since the data will already be there when we call them.

Asynchronous sockets have a particularly useful application when the program has other functions to do, such as drawing sprites, checking for other input, and so forth. There is only one catch with asynchronous sockets, and that is that since they rely upon the Windows Messaging queue, they are also exclusive to Windows. This is the kind of socket you would want to use on the client end of the application in most cases.

Setting things up

Since we will need a socket for every single client that is to connect to our machine, we will create a nice little structure to hold all of the basic client information.

#include <winsock2.h>

In this structure we have provided InUse, to represent wether this socket ( I will refer to it as "seat" in the future) is available or not. SOCKET will be the actual socket descriptor for that specific client, and soackaddr_in is used to hold the socket type, and address for the client. The two ThreadID variables will be used later on when creating a client thread so that we may control the thread later on.

We will also create a SOCKET and sockaddr_in for the listening socket:

CLIENTS     Clients[ MAX_CONNECTS ];
SOCKET      ListeningSocket;
Sockaddr_in Address;

Initializing WinSock

Unlike the BSD implementation of sockets, if we wish to use WinSock we will have to initialize and load the WinSock DLL. To do this we make a simple function like follows:

HRESULT InitWinSock( ){
  WSADATA  wsad;

  for( int i =3D 0; i < MAX_CONNECTS; i++ ){
    Clients.InUse = flase;
  }

  WSAStartup( MAKEWORD( 2, 2 ), &amp;wsad );

  return S_OK;
}

Essentially here what is happening is that we will create a variable wsad (of type WSADATA) to hold any information about WinSock that WSAStartup returns. From here we will cycle through all of the client structures and set them all as available and than we make a call to WSAStartup() to initialize and load WinSock.

MAKEWORD( 2,2), is just another way of specifying 0x0202, which to WSAStartup means load WinSock 2.2.

Should WSAStartup() fail, it will return non-zero. You should implement code to handle this even and call WSACleanup() should this occure.

Setting up the Listening Socket

Next what we need to do is take the ListeningSocket defined above and prepare it to bind to a port and start listening for incoming connections.

ListeningSocket = socket( AF_INET, SOCK_STREAM, 0 );

Address.sin_family       = AF_INET;
Address.sin_port         = htons( PORT );
Address.sin_addr.s_addr  = htonl( INADDR_ANY );

bind( ListeningSocket, (LPSOCKADDR)&Address, sizeof(Address));

listen( ListeningSocket, SO_MAXCONN );

What we did above was first set the properties of the listening socket. AF_INET pretty much means we will be using a standard TCP/IP protocol, SOCK_STREAM means that we will be using a guaranteed TCP connection as opposed to an unstable UDP connect (SOCK_DGRAM).

htons() is a function that will convert a normal number into a network short number, in this case the port number that we want to server to listen on, and htonl converts a normal number to a long network number. In this case s_addr is set to INADDR_ANY which specifies that we will listen on all interfaces. This is generally what you want to set it as.

bind will connect the socket to the specified port number, and listen will cause the socket to start listening for TCP_SYN packets - also known as your connection request. Note that SO_MAXCONN is used in listen(). This will specify the maximum number of connection to enqueue at once, anything more than this amount will be denied. Generally a value between 2 and 10 should work in here in most situations, while SO_MAXCONN is the ISP set maximum of connection requests at once.

Setting up for multithreading

What we will need to do next is create two functions, one of which will be used to start a client thread, and the other which will be the client thread.

HRESULT StartClientThread( ){
  // NO ONE ELSE CAN CONNECT UNTIL THIS THREAD IS READY
  ThreadInit = true;

  for( int i =3D 0; i < MAXCONNECTS; i++ ){
    if( Clients[ i ].InUse == true ){
      ClientID = i;
      break;
  }
}

Clients[ClientID].hThreadID = CreateThread( NULL, 0, &amp;ClientThreadEntry, 0, 0,
                                            Clients[ClientID].dwthreadID );

This is the code that will start the client thread. It has two variables which you will need to make global variables. ThreadInit, a bool, is used so that no one else can start a thread while we are still retrieving information on the one that is presently starting. This variable should be initialized as false. We also have ClientID, which will be the index of the Clients array that connecting client will use.

In this code we simply set ThreadInit to true, so that no one else overwrites our data before the client thread copys it, and use a for() loop to find the first available seat in which the incoming client will sit on.

After this we make a call to CreateThread which will in turn pass the address of the ClientThreadEntry function which will serve as our client's thread.

DWORD WINAPI ClientThreadEntry( LPVOID Arg1 ){
  int       ClientNo = ClientID;
  CLIENTS   *Client = &Clients[ ClientID ];

  Client->InUse = true;

  // MOVE THE SOCKET INFORMATION FROM SOCK A TO SOCKET B
  memcpy( &Client->ClientSocket, &SinkSocket, sizeof( SinkSocket ));
  memcpy( &Client->ClientAddress, &SinkSockAddr, sizeof( SinkSockAddr ));

  // DONE .. CONTINUE NORMAL EXECUTION IN THE PARENT THREAD
  ThreadInit = false;

  return 0;
}

This is the client thread, when the thread is started this code will be given its own point of execution and will only work with the one client that has connected.

I like to make things more convenient when I think of it, so I will create another Client Descriptor CLIENTS to point to the array item that this client connection sits on. I will also use ClientNo to store the index number to that array locally, remember that before the thread was called it was stored in a global which will be overwritten for the next client that tries to connect. We than set the client as in use using our pointer Client->InUse = true;

Now, before we can give control back to the parent thread to accept more connection we must copy the Socket, and sockaddr_in information. To do this we simply just perform a memcpy() on the global instances, and copy them into the client structure variables. Now they are ready for use by the next client so we set TreadInit to false and resume normal program execution of the parent thread.

When the function returns, the thread itself returns with that value and terminates so within this code a loop that would consistently checks for incoming data from the client would be needed. Once data is received, you simply check to see what is in the packet that was sent, and than make any changes that your require from that.

The next article will address receiving and decoding packets. I would write it with this article, but I am tired now and would like to get some of my own coding done.

Conclusion

With the code given above you should be able to write a basic server that will accept incoming connections and start a thread which will than copy the information into the client structure for its own use.

One thing you must remember to do is to clean up all pointers, closesocket( SOCKET ), on all opened sockets, and call WSACleanup() when your server shuts down.

Included with this article is a little program I wrote in linux to actually test the server. It will have more use for the next article, bur should provide a great demonstration of the Windows/Linux socket compatibilities.

In my next article I will address packet decoding and packet structures, as well as how to handle the different array of possible packets you could receive. I also intend to provide functioning cleanup code, and whatever else I can think of at that time.

If you have any suggestions or comments, please feel free to email them to me at cliffordr@hfx.eastlink.ca, or you can even come online on irc.afternet.org to channel #gamedev, #necrosoft, or #Toronto and talk to me in there since I spend 98% of my internet life in there anyway.