MMOs and Bandwidth

T1 would be efficient for a smaller MMO.

You would need a monster pipe to do real MMO activity like World of Warcraft, Everquest 2, etc..
All of those probably require an OC-3 at minimum.

An OC-3 for each quadrant of the world?

Quote:

Original post by I_Smell_Tuna An OC-3 for each quadrant of the world?

No, but linked up on a network. I don't think they have a seperate line for each server, but i think they have all of the servers in a network and then have one line going out.

When the user wants to go to a certain part of the world it would send them to a different server on the network. Then after that, it would send them to a different server.

The flow would probably look like.
1. User Connects to login server and login server verifies data provided by the user.
2.(Maybe) User connects to the Server Lobby(where it displays all the servers the user can play on.(this can probably be done without).
3. User is forwarded to one of the game servers which holds the quadrant of the world.
4. When the user enters a new quadrant, forward the user to a different server.

Edit: This is a BASIC flow. Obviously theres a LOT more inbetween such as databases and patching and such.

There are different ways of handling the servers, lots of different ways. Unfortunately I can't talk about them cause 1) I'm sleepy as hell, and 2) Agreements within my company as to non disclosure at this time :P. But anyway, there are other ways of spreading it.

There are multiple lines (OC-3) that are shared between dozens of load balanced servers. They might have something like 5 database servers (per site) that keep themselves in sync using replication, 2 or 3 servers running the same zone accessing the same data, etc...

When you login to Everquest, you hit the login server that resides at Sony. You then are asked to select a server to play on. Those servers are zone servers for the game in different locations around the world (Earth, not game world). They pass you around to the different zones in the game depending on your location in the game world. This is why your character in EQ is not transferrable between servers in the server list. This is EQ1. I am not sure if they changed this in EQ2.

It would be cheaper (and more practical) for a small company (like an independent) to use Shards. Ultima Online and Planeshift use this method. This means that the game runs on 1 server (plus a database server as always). UO uses a login server (similar to EQ) and gives you a list of shards to play on. The shards to not pass you to other shards though. The entire game is contained on one shard. So, in short, there are no zones.

what about grid computing based systems?

Quote:

Original post by paradoxnjThe entire game is contained on one shard. So, in short, there are no zones.

This is, in the case of Ultima Online, not at all true. Each "shard" consist of many sub servers, each running a specific part of the map hence the term "server line" Wich is the lines where these parts of the map meet. Funky stuff happens on server lines, monsters get stuck because they can't pathfind along a server line, extreme lag, where sometimes your backpack closes and/or you lose all your clothes and equipment for a split second while your character is in limbo between two map zone servers.

So, in a sense, Ultima Online has even MORE zones than opther mmorpg's since each (shard)zone is also split up into sevearal smaller (map)zones.

This is also noticeable during large public events where the EA servers are horribly underpowered and can't handle the load. Then that map sub server will be extremely laggy. And I mean freeze for 30+ seconds at a time, then walk one step and repeat kinda lagg... Whereas a few steps away on another map sub server everyting runs perfectly.

Regarding Grid computing: there are two uses of the word "grid": one is for geographically disjoint server clusters, and the other is just a synonum for cluster.

You want all server machines that talk to each other to be co-located in the same physical space; else the transmission latency makes life less pleasant. Thus, don't make them geographically disjoint.

When you build your own infrastructure, you may have to choose between a single provider, or multiple providers (more expensive, but more reliable). For each provider, there will be some kind of main feed (ATM, OC-3, what have you) that talks to an edge BGP router you set up. That router then typically goes into a switched fabric in your datacenter -- I can recommend Foundry Networks BigIron gear, although Cisco and the others also have good offerings.

When you co-locate in a high-class data center, the connection to other carriers is already taken care of for you; all you get is something like a GigE connection to their infrastructure. You still need some kind of router (BGP for large-scale systems, just plain IP routing for smaller) and a switched fabric for your actual machines to plug into. The connection you get will be rate monitored, and sometimes rate limited, so that you commit to a certain monthly average bandwidth (measured as the 95th percentile) and pay additional charges for overages.

The bandwidth use is very simple to calculate:

A. Peak number of online players (1000 for an indie, 100000 for a successful game)
B. Average amount of bandwidth usage (20 kbit/s per user, perhaps? depends on game)
C. Additional bandwidth usage (management, patches/updates, web services, etc)

The bandwidth you need is A*B+C. With a single T1, assuming you use 500 kbit for overhead (web, management, patches, etc), you have 1000 kbps to serve users, which translates to a maximum of 50 users. A small, indie game can probably get away with that. Also, if your game is an RPG (not very kinetic) you can probably push the average consumption down to 5 kbps or less while preserving a good experience, by trading off in your game design for minimal bandwidth consumption.

The demanding case is when 100 players all meet in the same place: you have 100 players that need continual updates of what all the other 99 players are doing; if each individual player stream is 0.1 kbps on average (about 100 bytes/second), that's 99*0.2*100 kbps, or the better part of a T1. For each additional player that joins the fray, you get both the additional stream of that player to every other player, AND the stream of all the other players to that player -- it's an N-squared equation.

Thanks for the info. Here's my next question, can an OC-3 be split into several smaller connections of equal bandwidth, each with their own IP address?

Yes, an OC-3 not only CAN be split, but regularly HAS to be split. Typically, you don't actually get OC-3 as an IP service; you get it as a transport service, and you have to arrange for a router on the other end, and a router on your end, to shuffle traffic back and forth. The OC-3 connects at a lower layer than IP, so what the IP addresses are of the data flowing back and forth is irrelevant to the OC-3 itself.

If you're actually planning on starting your own online business, I STRONGLY suggest you co-locate in a good-quality data center. And, to start out cheaply, I suggest you go for a few machines of self-managed dedicated hosting before you even get into co-location. Places like Server Beach or 1-and-1 let you get root/Admin on a machine you lease from them, and install and run whatever software you want, with some capacity caps per month per machine.