MMO Networking

The Server will be written in Java and the Java.Net API will be used for the Networking part. What I can not find are any good Tutorials or Articles about Networking for Massive Multiplayer Purpose.

MMO networking isn't that different from a regular networked game, except you have to deal with even more separate entities. You can start with a FPS-style state-update system, and turn down the update rate (prioritize it based on distance to viewer) and you'll have a good first cut.

However, I'd like to contribute a little bit of a reality check:

[color=#1C2837][color=#1C2837]

Im already an advanced Java Programmer with 3 Years practical experience.[/quote]

[color=#1C2837]
[color=#1C2837]

It generally takes 10 years to become "advanced" (as in "expert") in any particular area. I don't doubt that you can program in Java, I'm just trying to help you set expectations.

[color=#1C2837]
[color=#1C2837]

Good luck on your project!

I would recommend the common method of TCP/IP for login and initial connection, then transition over to UDP for gameplay. Typical FPS connectivity is advised because it is based around an authorative server which is a must for any MMO. The rates of update probably do not need to be as high as with an FPS, but if you have PVP then you will need a reasonable rate. What you will want to do is use a normal client-server prediction system which you can find threads about here. You will also want to partition your world somehow and have it generate snapshots for each partition with players in them. Then it can just dump the area snapshot to all players within that zone instead of trying to create custom visibility based snapshots per player. You could further improve snapshot performance on a very busy area by updating hostile elements in the snapshot more than friendly elements to allow a person to get informaiton that could save them.

While many people assume and recommend using UDP for MMO's keep in mind WoW uses TCP/IP and last I checked it runs very well. Don't get fooled into thinking that UDP is the way to go for MMORPG's. However good luck with your project!

UDP is a very bad idea for an MMORPG, since people are gonna be pissed, if loss of packets is gonna cost them an item. Plus you still want to have certain ensurances by your protocol that you end up implementing yourself. Never optimize early! I also recommend a single server architecture to start with. I come from run-dead projects that never made it off the ramp and I'm telling you: You want to keep it as simple as possible, so you get something to show for, as early as possible. That keeps motivation up and starts producing a user-base. Don't do it quick and dirty, keep encapsulation and modulation on high levels, but don't get into the dirty details that can easily be added at a later point in time.


As was said before, the basic server framework just has to serve clients fast enough. Make sure, you have a smart threading model that can very well facilitate multi-threading and never does I/O on any world thread. If your threading model works and you have no hard-core bottlenecks, you'll be able to easily support hundreds of players. You might want to check out and use Erlang's threading model as an example. Finer-than-fine-grained levels of concurrency without synchronization issues, at the cost of delayed remote code invocation adds a bit of complexity but makes for fast execution without stalls or starvation which is the key in any server architecture.

When serious robustness is a requirement, then it is not a question of whether to use UDP or TCP. You will need to use both. You won't use them for a single client simultaneously, but allow clients to connect using either.

The reasoning for this is that for latency-related robustness, TCP cannot beat UDP, because TCP forces ordering, whereas with UDP you can stream small messages with latest data guarantees. On the other hand, for connectivity-related robustness, UDP cannot beat TCP, since sadly, there are lots of network firewall infrastructures which do not allow UDP connections at all (and most likely limited TCP port ranges).

Therefore it is a good idea to abstract your network communcations in such a way that you can connect through either UDP or TCP, and get the best of either of them (and not the lowest common denomination of both ). It is not that hard, since crafting messages will work pretty much the same way, independent of whether you are going through UDP or TCP.

There are also several existing networking libraries, raknet, enet, lidgren, knet (link in my sig) you can use.

UDP is a very bad idea for an MMORPG, since people are gonna be pissed, if loss of packets is gonna cost them an item.

Lost packets end up meaning lag, no matter whether you use TCP or UDP. If a packet is dropped, TCP doesn't magically get the message to the destination any faster than UDP. The only defense against "lost items" is proper game design and rules implementation.

never does I/O on any world thread
[/quote]

This is very good advice, though. Specifically, the design that falls out of this advice is that all your dependencies (disk, network, etc) have to be asynchronous, so you never end up blocking in any world/rules/per-step thread.

Lost packets end up meaning lag, no matter whether you use TCP or UDP. If a packet is dropped, TCP doesn't magically get the message to the destination any faster than UDP. The only defense against "lost items" is proper game design and rules implementation.
[/quote]
Of course, but a lot of development overhead is required to make sure that packets are not randomly lost and the game still can function correctly.
As I said,

you still want to have certain ensurances by your protocol that you end up implementing yourself.
[/quote]

If you don't have very similar safety features as TCP, you get intolerable error probability and if you don't have a solid background in networking, you always end up having to cope with bugs that arise from the fact that you did not properly consider uncertainty, packet loss, invalid ordering and some such.

As was said above, WoW uses TCP only and even single-threaded private servers (plus one thread for I/O) have acceptable performance and no TCP-related lags with hundreds of players online. Don't forget, it's an MMO and not an FPS - 10 ms faster response time is not worth getting people upset over losing their precious bits and bytes that represent their online life.


Of course, for one-shot streaming data, such as voice chat, you can use UDP without having to overthink things.

As was said above, WoW uses TCP only and even single-threaded private servers (plus one thread for I/O) have acceptable performance and no TCP-related lags with hundreds of players online.

I would gladly support this in the past.

But a few months back, for one reason or another, Blizzard found it fit to give me a free month of WoW. So I decided to check what happened since BC. But it also coincided with me having some ISP networking problems, resulting in constant 2-5% packet loss.

In-game latency I had was anywhere up to 5 *minutes*, including no session lasting more than 10 minutes. I also think that WoW doesn't particularly try to correct for packet loss issues and just keeps streaming reagardless until client overflows and is disconnected.

So no, TCP doesn't work if there's even minimal non-transient packet loss.

10 ms faster response time is not worth getting people upset over losing their precious bits and bytes that represent their online life[/quote]

A client may never be authoritative, so even if it does lose some information and is never told about loot, it will still be correct on server. Worst case, user's UI desyncs.

And besides, the lossiness of protocol is typically well structured around some well-defined and understood concepts (state, movement, chat, trade, actions) and encoding that into protocol handler or application logic isn't all that big a deal.

For FPS, most of complications tend to arise from prioritization to optimize latency and bandwidth.

a lot of development overhead is required to make sure that packets are not randomly lost and the game still can function correctly.

It's not *that* much.

If you don't have very similar safety features as TCP, you get intolerable error probability

Depends on what probability of error you can tolerate.

and if you don't have a solid background in networking, you always end up having to cope with bugs that arise from the fact that you did not properly consider uncertainty, packet loss, invalid ordering and some such.

Certainly true.

Of course, for one-shot streaming data, such as voice chat, you can use UDP without having to overthink things.

As always: choose the right tool for the job.

I would gladly support this in the past.

But a few months back, for one reason or another, Blizzard found it fit to give me a free month of WoW. So I decided to check what happened since BC. But it also coincided with me having some ISP networking problems, resulting in constant 2-5% packet loss.

In-game latency I had was anywhere up to 5 *minutes*, including no session lasting more than 10 minutes. I also think that WoW doesn't particularly try to correct for packet loss issues and just keeps streaming reagardless until client overflows and is disconnected.

So no, TCP doesn't work if there's even minimal non-transient packet loss.

Sorry to say but you having networking issues (which you mention) does not make "TCP not work" with "delayed" messages. It just didn't work for you when your ISP was having issues, which you mentioned. Unless you know exactly what was happening to the packets leaving and coming back to your pc, making such a bold statement is silly.

In-game latency I had was anywhere up to 5 *minutes*, including no session lasting more than 10 minutes. I also think that WoW doesn't particularly try to correct for packet loss issues and just keeps streaming reagardless until client overflows and is disconnected.

To be fair, this was a "sample of one."

A packet drop rate of 2-5% is pretty bad -- way more than any "normally" operating ISP should give you.
However, TCP should still work alright.
The fact that it didn't,may either depend on the packet loss being even worse than that, or the extremely bad gyrations in RTT estimates you get from buffer bloat somewhere on the line.
No device routing TCP should keep more buffer than half the RTT of the next hop, in the ideal case. However, for most home routers, this translates to maybe a handful of kilobytes. Embedded chips used for routers these days have RAM measured in the megabytes, so it's easy (and common) for designers to over-provision the buffering. The end result is that the sending end thinks the pipe is fatter than it is, building up a large buffer of data, then finally getting a very long duration of almost-total packet loss, which slams the throughput estimation down to almost zero. After a while, the throughput will build up, and start oversubscribing the pipe, building up a buffer, which finally generates another window of near-total packet loss...

Or, who knows. They maybe had other issues. Hard to tell without a packet trace on both ends...