🔒 Locked

Server implementation...

Started by Imgelling May 23, 2007 at 12:23 AM 14 replies 4.4k views

Original Post

May 23, 2007 at 12:23 AM

Hi, let me start off with I am using c# though I don't think it matters in this query. Plus what I am attempting is a simple chat program and am using TCP. In my server, I have 3 threads. A=Connection Thread:Uses accept to allow users to login at anytime of server operation B=Select Thread:Uses select to find which sockets in an ArrayList have data to receive C=Receive Buffer Processor Thread: Processes the receive buffer Now, for 2 out of 3 threads, B and C, each has methods associated with them. For B, is the Receive method which separates the incoming streams into packets. Then stuffs those into the Receive buffer; an ArrayList. For C, the methods corresponds to different packet types (Packet processes) and for each packet process, corresponding methods to deal with separate versions of each packet type. I realize C seems a bit confusing, so here is an hierarchal view


Receive buffer processor
     A: Error Packet 
          1. Connection reset by host forcibly
          2. other errors
     B: Text Packet
          1. Global text
          2. Private text

Each alphabetic list is its own function as with the numeric list. There are others but that should suffice. Each thread runs full speed (well, with a Thread.Sleep(0) after each iteration) using Synchronized ArrayLists. No "heartbeat" or "Process only 20 packets per iteration" implemented as I have not (yet) had a need to do so. Is this a reasonably good implementation or is there another route I should take? I don't want to run into a wall down the road that will cause me to do a re-write. So I thought I would get some opinions on what I have so far. Thanks for any insight! EDIT: As a separate question, is a synchronized ArrayList good enough or should I still use locks for the data? http://msdn2.microsoft.com/en-us/library/system.collections.arraylist.synchronized(VS.71).aspx Says its thread safe, but I just wanna hear some opinions about it. [Edited by - Imgelling on May 23, 2007 12:47:54 AM]

humanapp

May 23, 2007 at 2:24 PM

Imgelling,

I would feel more confident offering my opinions about server architecture were you implementing your server in C/C++, but C# remains a hobby language for me. That said, I think I'd explore the asynchronous socket support the .NET Framework has to offer before going the more traditional route. They seem to have taken much of the pain/confusion out of using I/O completion ports.

Here's a link to an asynchronous server socket example in C#:
http://msdn2.microsoft.com/en-us/library/fx6588te(VS.71).aspx

Imgelling

May 23, 2007 at 3:54 PM

Quote:
Original post by humanapp
but C# remains a hobby language for me.

That is sort of why I am creating this server. To prove to my fellow programmers here in the area that C# is more than adequate for a server. My plan is to start out with a simple, console chat program. Then to a text based MUD using XML (or SQL, haven't decided yet) for the database. Finally on to a simple graphical (2D,XNA) MUD using the fore mentioned database and C#'s reflection abilities for scripts.

As for the asynchronous sockets, I began using those and found them to be problematic. Of course I knew nothing of networking at the time, so I guess it wouldn't hurt to take a second look.

Well, something I just thought of, wouldn't the architecture still be the same, just no threads (well unless I want to keep the receive buffer processor in its own thread)?

Antheus

May 23, 2007 at 4:35 PM

Quote:
Original post by Imgelling
Quote:
Original post by humanapp
but C# remains a hobby language for me.

That is sort of why I am creating this server. To prove to my fellow programmers here in the area that C# is more than adequate for a server. My plan is to start out with a simple, console chat program. Then to a text based MUD using XML (or SQL, haven't decided yet) for the database. Finally on to a simple graphical (2D,XNA) MUD using the fore mentioned database and C#'s reflection abilities for scripts.

As for the asynchronous sockets, I began using those and found them to be problematic. Of course I knew nothing of networking at the time, so I guess it wouldn't hurt to take a second look.

Prove to whom? In any serious debate the usefulness of C# isn't an issue. The reason some people dislike C# has to do with either development culture, legacy compatibility or other business related reasons - not with the language.

Also: async sockets, unless I'm mistaken, use IOCP in their implementation. Your problems might stem from being unfamiliar with asynchronous programming, not the sockets themself.

Main thing is, no matter which serious language you use today, the performance increases that have occured during recent years aren't because of better languages, but because of increased hardware and operating support.

A server written in VB today can and will be just as effective as one (ok, almost) in C#, C++ or assembly. It will rely on OS calls, and OS support has improved a lot. Things like lockless primitives help with that as well.

Personally, I don't even think about writing synchronous IO for anything anymore. It may be an overkill in some cases, but it's not really that much more complex.

If you're going for scalability and performance, asynchronous is the way. No other aproach will come close, since that one has native OS support on just about any modern OS today.

Imgelling

May 23, 2007 at 5:02 PM

Prove to some of my programming buddies here in town, not on Gamedev. Sorry if that was misleading. They seem to think C# is only for "noobs", as they put it, that don't want to learn C++. I have even seen, tested a MMO server that was written in VB. Worked quite well.

Well thanks for the insight. With your input and humanapps, I have decided, the best thing would be to go with async. sockets. Shouldn't be too much of a problem converting over, my actuall socket handling code is pretty much seperated from the rest of my code. And since I have gotten this far with blocking calls, I should be able to better understand the non-blocking calls.

But back to work, boss is here.

Ozymandias43

May 25, 2007 at 11:24 AM

C# is for n00bs that don't wanna learn C++.
C++ is for n00bs that don't wanna do object-oriented programming in C.
Object-oriented C code is for n00bs who can't keep their procedural C code in order.
C is for noobs who can't handle assembler.
Assembler is for noobs who can't handle machine code and a hex pad.
machine coding on a hex pad is for n00bs who can't grok functional programming in Lisp.
Lisp is for dinosaurs that don't wanna learn ML.
ML is for dinosaurs that don't wanna learn Ruby.
Ruby is for people to idealistic and establishment-rejecting to use Java.
Java is for people too anti-Microsoft to use C#.

And the circle of life continues.

kalannarDevir

May 31, 2007 at 12:12 PM

Getting back to the question on implementing a server. I would recommend using a library like Lidgren.Net. You will save a lot of time in the long run when you start programming your graphical mud.

The way I have implemented a game server in C# is to have a connection manager and a game manager running in different threads. The connection manager just receives and sends packets. The game manager handles the logic.
When a packet is received by the server, the connection manager places it into a receive queue. The game manager then dequeue the packets and handles the logic. When the game manager is ready to send packets, it places them in the connection managers send queue.

The example above is simple, and for a more complicated game I would use more managers and threads. I am working on a server in C# right now. Send me a private message and we can exchange emails. I will be happy to exchange knowledge on this subject.

hplus0603

May 31, 2007 at 1:06 PM

Quote:
is a synchronized ArrayList good enough

If each operation is done using the synchronization, that's true.

However, it's not clear how you would do the following operation in a thread-safe fashion:

Check whether there is anything to removeRemove and return the first item

Using .NET code, you'd have to write it like:

item = null;if (array.Count > 0) {  // consider pre-emption HERE  item = array[0];  array.RemoveAt(0);}

Those are three separate operations on the synchronized arraylist wrapper, and thus there is still a race condition between threads. I would instead use a non-synchronized container, and use a separate lock, which lets me wrap that entire code snippet in a single critical section.

Consider if there is one item in the list, and the first thread gets to the point of the comment, and then is pre-empted. The second thread runs the same code, removes the last item (leaving the list empty). Then the first thread is switched in. It will now try to get the first item of an empty list.

Similarly, you can get into a situation where two threads want to operate on the same item, although one of the threads will try to remove from an empty list, by having the pre-emption happening right before the Remove.

Imgelling

Jun 1, 2007 at 3:24 PM

As of currently, I still have not switched over to async operations. The only place the synchronized array is being manipulated it in the select thread and receive buffer processor thread.

I had a problem with exactly what you are talking about. At first, when a client gracefully disconnected, I would get a disconnection packet from them and stuff it into the buffer. Then before the receive buffer could process, the select thread would run 1-7 times saying the same client had data to read and returned 0 bytes (of course meaning the client has disconnected). So when the processor thread ran, it would read the disconnect packet. Then after removing the user socket and the packet from their respective synced arraylists, it would move on to process the "client disconnected non-gracefully" packets and would cause an error, because the user has already been disconnected.

To fix, in the select thread, I immediately remove the user from the socket list and send the socket with the packet to the processor to process. Works fine most (like around 99.9%) of the time, but every once in a while, I will get an ungraceful disconnect packet immediately after a graceful packet. I also implemented Thread.Sleep(0) into all threads.

But as soon as I get some time, I will convert my code into asynchronous operations, thus removing the threads all together. The receive processor will be moved into the main thread, called every so often. And data locks will be implemented.

Thanks for your input Hplus!

kalannarDevir: PM sent! Yeah, about a month late, but I am still working on my server. Not sure if you are.

cwilson

Jun 1, 2007 at 5:38 PM

Imgelling, given what you have said you're trying to accomplish, your design is adding considerably more complexity than you need. If you just want to demonstrate that you can write a workable chat server in C#, *and* you're relatively new to network programming and/or server programming (which is what it sounds like, but apologies if I'm mistaken), you'll probably be better off getting it working as a single-threaded app first. Benchmark your initial implementation to see where the bottlenecks are, and then maybe make it more complex. You'll probably find that your single-threaded implementation works just fine for what you're trying to accomplish (particularly for chat). Plus, you'll get more core logic done because you won't be spending as much time chasing down hard-to-reproduce race conditions.

Just have your main thread run a select() loop (or C# equivalent), and do your accepts/sends/receives from within the loop. This is really all you need to do for now. Having three threads the way you have it now (especially with the sleep(0)) is almost certainly hurting overall performance, depending on your hardware configuration. But in any case you're basically polling on two of the threads which is chewing up lots of CPU cycles unnecessarily.

Once you get it working, and become more familiar with the overall principles, then you can adjust the code as necessary for better performance (but only if you need it!), e.g. using worker threads or perhaps asynchronous io.

Having said all of that, if part of this is just to serve as a learning exercise, and for example you're making it multi-threaded in order to learn more about using the threading APIs, then of course you can make it as complex as you'd like. :)

Imgelling

Jun 2, 2007 at 5:50 AM

Hi cwilson. Guess I should have stated somewhere that this is not my first attempt at a chat server. But it is the first time to use threads, and after actually posting this, I have learned I am using threads when I don't need to (basically what you said). And yes, you assuming I was new at networking was correct, but I'm getting there.

Also, as of about 2 weeks ago, I have finished my simple chat server.
Includes but not limited to
* User name/password use that is saved and loaded.
* The user list can be edited at run time on the server
* Stat recording, but the stats are only bytes sent/received
* Up to 64 simultaneous users (not tested and limited only by Socket.Select in theory). Tested up to 5 users on same machine.
* A simple server side menu which allows seeing who is online, said stats, saving/loading user list, adding users, and broadcasting a message to all users online. On the other hand, this menu has been programmed in the MOST simplest way but it works.
* Handles a website testing to see if the server is online. The code is not something that you would probably recognize from the rest, but it would crash if I did not include it.

Now, on to the simple text MUD using the helped and insight gained here and in the programming of the previous stage of the server. Right now I am brainstorming on what I want to include, how I want to handle the database, and skimming through the book "MUD Game Programming" for some ideas. Plus I'm sure I will be asking a question or two here in the near future.

I would like to thank everyone for their opinions. If anyone would like to actually see my code, I am more than happy to share. Just message me at Imgelling@gmail.com.

Comments on the code...
The code is not documented very well, but I am willing to jot down a few comments in the code if requested. The code is also not very organized, but its very readable.

Thanks again.

cwilson

Jun 2, 2007 at 10:04 PM

Well, it sounds like your application has already outgrown my earlier suggestions. I would only add that if you're now hitting a database (for registration and auth), the single-threaded implementation is sub-optimal, because the main loop will block on database calls. If you haven't already, you might look at implementing a worker thread pool to handle incoming requests.

Good luck. Have fun! :)

- Chris

elFarto

Jun 3, 2007 at 4:17 AM

Hi

You may wish to look at the 'Reactor' design pattern. Here's a link describing it, it's in Java but it should be easy to convert to C#:

http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf

Regards
elFarto

kalannarDevir

Jun 3, 2007 at 9:44 PM

Imgelling I sent you a PM. I have completed a text based mud server. I am now working on a graphical mud server.

kalannarDevir

Jun 5, 2007 at 1:47 PM

Imgelling, I sent you two PMs, if you do not receive them, then PM me your email, and I will respond to it.

humanapp

Jun 6, 2007 at 8:48 PM

Have you considered using the generic Queue class? It's FIFO. Here's some pseudo C#:

    Queue<NetMessage> pending = new Queue<NetMessage>();    // Gather pending messages inside lock    lock (queue) {        while (queue.Count > 0)            pending.Enqueue(queue.Dequeue());    }    // Dispatch pending messages outside lock    while (pending.Count > 0)        DispatchNetMessage(pending.Dequeue());

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.