MMO Client == Server

First off:

I'd prefer not to see reasons why this can't work[/quote]

Don't say things like that. If the only thing you ever want posted are the Pros to something, then you aren't going to get an unbiased picture of anything. Sticking your fingers in your ears and going LALALALALA doesn't work well .

Past that, you've described a Peer to Peer MMO. There is one caveat that I notice in your description: You say that it would have no central server. But it does. You explicitly state that there is a central database. No matter what, you are communicating with said central database.

Now, you are going to run into a HUGE issue that is relatively insurmountable: you will find it very difficult if not impossible to prevent cheating. Modern MMOs use server-authoritive systems because the _server_ then decides what is valid. When you allow clients to do so, they can cheat and it is very difficult to verify. Are you going to let the other clients verify it? They can lie, too.

Past this, you are actually looking at MORE network traffic per client, but less to your central database (as it isn't authorative).

Also, how do you plan on propagating information from clients that are not directly connected? Is it going to be through the 'central database', or is there going to be some sort of web of connection? If the former, just implement a server-authorative system; you're halfway there without the benefits of it. If the latter, how are you verifying that all client nodes are connected to one another? Depending on the complexity of the connection tree, some clients may see relatively high amounts of latency.

One thing to remember is that when an average user's computer is connected to the internet, it potentially has
1. a firewall and/or a restrictive proxy software running on his computer
2. a firewall and/or a restrictive proxy running on his home network box, local network, or company network
3. a firewall and/or a restrictive proxy running on his ISP's routers
4. a network address translation (NAT) layer on his home network box
5. a (second) NAT layer on his ISP's routers

All these prohibit you from spontaneously connecting direct peer-to-peer connections. The issue nr. 1 is sidesteppable by appropriate firewall exception configuration, the issues nr. 2 and 4 are sidesteppable by a knowledgeable user, but basic users don't understand how to do this. The issues nr. 3 and 5 might be sidesteppable by agreements with the ISP, potentially having money involved. Of course, you can't negotiate with ISPs for the customer's behalf, and you will generally need to be able to solve all of the items above.

The general techniques that are employed by peer-to-peer systems today to solve these are:
- NAT punchthrough. ([1], [2]) This is not possible with TCP for security reasons, but you will have to use UDP. NAT punchthrough will require a centralized arbiter to hand off connections.
- Masquerade into known ports, like TCP 80. Some firewalls allow inbound connections to these. Skype extensively utilizes these to penetrate network firewalls.
- Fallback to using centralized server architectures when neither of the above works. This might sound disappointing, but it is the only way to provide a robust system which guarantees 100% connectivity.

Skype and various Torrent architectures provide interesting case studies on how creating connections "from the internet" towards a single client can be handled.

Networking has always kind of been my weak point, so i'm sorry if don't sound like i know exactly what i'm doing.

I'm sure if this has been done efficiently, it would be quite popular, but i don't really know where to look for any sources if there are any, or if this technique has a name or not, so i'm kinda wondering if there are any sources out there that might make this easier. If not, it's still a long way down the road before i begin working on the networking specific tasks for this project.

I'm working on an MMORPG right now.

This is a mistake. Any multiplayer project's design is deeply connected to its network behavior. I would recommend thinking about this sooner rather than later because otherwise you risk having to undo a lot of work to get your network implementation solid, and simultaneously risk breaking the game severely if you aren't painstakingly careful.

So what i'd like to do is to have a client connect to other clients instead of a central server. There will still be a central server which contains the database for the game, and do other tasks like updating the client software, connecting the client to the other clients and whatever.

What does this gain you?

What it loses is pretty substantial:

- Cheating will be trivial
- Players will desync with each other left and right
- You will create a vast amount of work for yourself
- Network traffic requirements will go up by a large factor
- Connectivity issues as clb outlined will make live very difficult

And I'm sure there are more negative consequences that would crop up as well.

I've put a little thought into this, like having the client connect to 3 other clients or so (in case one or two disconnect), choosing the clients depending on their ping (out of a group of random clients so they don't have to ping every connected player). When one or two of the 3 connections disconnect, the other one will send the data to the central database to be stored. This way the client will be receiving the same packet 3 times (one from each client it is connected to), instead of the 1 from a central server, but it also means that certain tasks could be done not on the client, such as collision detection.

I still don't understand what this buys you. Your game is now less reliable, more laggy, easier to exploit, and more frustrating to play for people behind firewalls/routers. There seems to be no positive side to this at all.

Consider this scenario: players 1, 2, and 3 are connected peer-to-peer. Player 4 enters the world and starts playing in the same area as 1, 2, and 3, but player 4 is connected to players 5, 6, and 7 instead of 1, 2, and 3. How do 1, 2, 3 see player 4? Either they can't (in which case your "MMO" is really not going to be very enjoyable) or you have to relay through potentially dozens of hops before you can get updates from 4 to players 1, 2, 3. Latency is going to shoot through the roof and desynchronization will be rampant.

Of course this would open up a world of hacks, but they can always be minimized by checking the data that's been sent to the server with data from the other one or two clients. It would also allow the central server to spend more of it's time doing checks like this.

So all I need is two (or three) clients to tell the same lies, and I've broken your protection.

I think you're misunderstanding the work load on the server as well. Instead of just simulating the world once and verifying that all clients are in line, you have to simulate every client because they will drift apart to some extent because of your latency-multiplying peer-to-peer design. So instead of one world sim for everyone, you now need to simulate one world per client on your server. There is no way on earth that is going to be a net gain ;-)

Any ideas on this? I'd prefer not to see reasons why this can't work (without an explanation, but more importantly, what would need to be done to make it work). Be productive and try to take it easy on me

There is a reason nobody does this - because it can't be made to "work" in the sense of actually doing what you want, being robust, being responsive, and not having huge negatives. It's not because of some secret cult that loves servers and assassinates people who try to think for themselves and defy the server-centric gods ;-)

haha! thanks for the responses, they were very good points, although the outcome of this idea isn't looking too bright anymore

apoch, i know your right, i need to be thinking about the networking now because it's very important to the structure of the game, that's sort of why i'm asking about this now. i'm not ready to implement any server client stuff yet, but i'm making sure to design the game so that data can easily be sent, received, and processed, just not sure how i'm going to be sending or receiving them yet.

i wrote this when i was a little tired, so when i said "I'd prefer not to see reasons why this can't work", i didn't mean i don't want to see reasons why this can't work, even though that's exactly what i said, i just meant that if this WAS to work, what would need to be done, of course, the cons of it will need to be known so we can fix them to make it work. i didn't mean to sound rude, sorry. I'm thinking this probably isn't going to work after all, but after thinking about it some more, i think i can describe it a little better.

First, connecting to 3 other clients won't really work that well anyway, since that means every client HAS to have at least 3 people connecting to them as if they were the "server" and connecting to 3 other clients as if they were the server. that's 7 connections at least (1 for the central server). I think saying server is also the wrong thing to say. Of course, there will be a central server so people can see each other and whatever.

so anyway, one more go at this, even though i see it's probably a lost cause. What if every client connected to another client, so that other client can do things like collision detection on their computer, then send the result back. like, if your playing the game, then all the rendering and drawing are processed and done on your computer, but when you give input, that input is sent to the client you are connected to, where it is processed. So all the rendering is done on your own machine, but the game processing is done on someone else's machine. like cloud computing right?

I know that would open a new door to cheating, but if you don't get to choose the client you are connecting to, and the server can verify you are connected to the client it gave you, then minimizing hacks would be the same as a regular client/server connection, except that now you wouldn't be able to cheat for your own character unless you got lucky and you have two machines and somehow you randomly got connected to your second machine. the less people playing the game, the more likely chance that this could happen, but you would still need to figure out how to hack the game itself, but that's a different topic i think.

really only the input is sent to the other client, who is connected to the central server on your behalf, then the result of that input is returned to you. frame rate would depend on the connection to that other client. I really think this would minimize cheating if done right. you are still connected to the central server though, for the person who is connected to you for you to do the processing for them, and for any other reason you need to be connected to the server that doesn't have to do with your game character or whatever, like maybe if you get disconnected from the other client, the server can send you a new connection.

I know this probably won't work, and the lag might be terrible, but any more ideas?

I still don't understand what you think this will gain you...

if all the game processing is done off the clients machine, doesn't that make it nearly impossible for him to cheat? originally i had the idea that if players could use the network of other players as the server, then the main game server wouldn't have to do anything really but hold a database, update client's software, and connect them to other players, and act as the little glue that keeps the players all together. I also know it's not too difficult to make a hack for a game so you are able to walk through walls fake collisions or bypass a collision check, such as when players attack each other

dos attacks could be a huge problem, but it wouldn't really gain the player anything by doing it, since he would freeze on a frame while he's waiting for a response from the client he's attacking

i guess another big problem is that since everyone is connected to one other client, then if there's an odd number of players, someone will have to have at least 2 people connected to them, and do the processing for both. not to mention when a client disconnects that had someone connected to him, then that other person will have to wait for a new connection with someone else, and for that new connection to load all the geometry that needs to be checked for collision. maybe it's really not a good idea... haha

maybe people could get extra credit for the time they leave their client application opened, so even if they are not playing the game, another player could use their system resources to do all the processing.

Cheating can only be increased by giving any client authority. The client might not be able to choose to run at twice speed, but they could make their peer run at half speed. If you peer clients by location, then cheating "clusters" like a clan could easily end up simulating for one another, with obvious consequences.

By giving any authority to the client, even if it doesn't extend to the player using the client, introduces scope for cheating. What about teleporting the peer close to the client, and then abusing the simulation to defeat that player on that peer and steal their loot?

I don't know how this concept would extend to situations where large numbers of players gather nearby. It would be extremely complicated to get a consistent simulation over internet latency between the systems. Which player dealt the killing blow to the dragon? Jimmy's peer claims his axe removed the last few HP, but Jane's peer is adamant that her Lightning Bolt of Hurtiness did the final damage.

I think you need to create a list of pros and cons for this idea. If you are honest, you'll find the pros are fewer than the cons, and the cons have much larger implications.

I know i probably shouldn't keep this going now, but stopping a person from teleporting his peer close to him would be the same as stopping him from teleporting himself if all the processing for his character was done on his own machine.

it's true that giving the clients any authority is grounds for hacking, so my original idea looks settled, but i still think if all the processing that would usually need to be done on someones own machine is done on someone else's machine, it would minimize cheating even more. the clan thing is a good point, but even then, if they were able to cheat for each other, they would be able to cheat anyway if the processing wasn't done on their own machine.

i've decided not to put any effort into doing something like this, but if you have any more good points, i'm still interested

You're still mistaken about the architecture actually decreasing cheat potential.

It makes it a little more interesting, but frankly, all that's going to do is draw better attackers to the challenge.



Consider this scenario:

- Player A is connected and playing, with his simulation running on Player B's machine
- Player B decides he hates Player A
- Player B turns on a tool that starts reporting bogus information about the simulation to Player A
- For instance, Player A might now be told he's walking through lava and lose 5 HP/sec for forever

You've made it a little more awkward to cheat directly in your own favor, but even that isn't all that big of a deal. What you're actually doing is making a game that is prime spawning grounds for griefers.

It's still totally possible to cheat in your own favor once you know how the system works. For instance, all you have to do is tell the right lies to the right clients. Consider a PvP mode of your game:

- Player A is innocent and hosted by Player B
- Player B is malicious and hosted by Player C

Since Player B controls A's simulation, he simply tells C that he's always 2 inches behind A, and constantly firing off laser beams into the back of A's skull. C has no way to verify this under your proposed scheme, so C believes it. Somehow (via some magical mechanism you've declined to elaborate on) A then finds out that he's dead, because C's simulation said so.

Like I said, you're really not actually improving the situation. You're requiring a little more creativity from your attackers, but the first lesson of security is as soon as it's a challenge more people will try to attack it - and there will be some very smart people in that group.


Now, suppose your game has an economic side. With your peer-to-peer topology, I guarantee I could permanently devastate your game economy in a matter of hours at the most, maybe less if your game implementation is sloppy enough. All I have to do is lie to people to try to sell items while I'm their host, and refuse to let people buy items at the same time.

You should always assume that the attacker has 100% of your source code and can forge any network traffic he pleases. It's basically true whether you like it or not. And as soon as you open up any compromised computer to being an authority on the simulation, that compromised machine can do whatever it damn well pleases with your simulation's contents.


Think of malicious attacks like herpes. You don't defend against herpes by "spreading the love" to as many people as possible. That just gives you more infected people to continue propagating the disease.

What if no single client is authorative over any single other client, but rather a majority rule by all of the connected clients? B can say A's dead all he wants, but if C, D and E don't think so they can ignore it and not propagate it through the rest of the network. A can say he's moving from X to X+1000, but even if B agrees, C, and D all know this is impossible and refuse and since 2/3 clients say no it doesn't happen. This won't make it impossible to cheat, but if the central server makes it hard for a clan to ensure they're significantly networked it'll make it harder to do so.

That still leaves you open to griefing easily. All I have to do is make the client randomly reject other people's packets and suddenly they get a laggy experience because their updates aren't being accepted uniformly.

You also just raised the computational requirement of the game by a huge factor. Suppose you need 3 out of 4 machines to agree in order to "accept" a movement. If there are 16 players connected, you have 4 different "groups." Each group must now simulate 5 players (themselves + their quota of 4) instead of just 1. Across the entire system, you've increased the performance cost of the simulation without actually gaining anything, and now you have a distributed computational problem to solve, which is substantially harder than just a client/server setup.

The more users you require to vet a player's activities the higher the cost goes and the more latency you have to tolerate to wait on a quorum.

Also, all it takes is one player on a slow connection and now several people have bad experiences, because that client will slow down the agreement process. (This is the "accidental" flip side of the griefing method I mentioned.) So in essence you've made your work harder, chewed up a lot of CPU for no good reason, made the game perform worse, and still haven't gained any advantages of any kind.


I don't really understand people's fascination with this; I'm still curious what the perceived benefit is :-)

I don't really understand people's fascination with this; I'm still curious what the perceived benefit is :-)[/quote]

I think there are two possible ways you can end up there.

1) "Hey, playing MMOs is fun! But, oy, the monthly fees cost my entire allowance! I bet the reason it costs so much is all those servers and bandwidth -- if a MMO could be built without servers/bandwidth, it would be fun and free. I will make the world a better place!"

2) "Hey, I want to cash in on the MMO craze! I bet I can design a guild/crafting/player-housing system much better than what all those other game designers have done! But I can't afford to lease servers on the meagre dollars I get from my paper route. I'll just design a system where all the players are both servers and bandwidth providers. Soon, I'll be rich!"

Both of those approaches stem from a basic misunderstanding of the relative difficulties involved in bringing a MMO to market. Building all the content, and designing a game that will actually attract and retain players, is *by far* the hardest challenge. By something like two orders of magnitude. The technology is usually pretty well known by know. If you actually want to pay for experienced engineers -- if you roll your own, well, you're a pioneer where everyone else has already been!

Once you have a game working, leasing some servers to bootstrap the game is not a problem. Save your coffee money while building all the content, and once you're done, you'll have the $100 it takes to lease a dedicated server with 2 TB of bandwidth for two months (interserver.net?) That should be enough to get five players to sign up for $10/month -- and if it isn't, well, you probably shouldn't keep operating that game anyway :-) Not to mention that, for starting small, you can start with something like a $20/month virtual server (Linode?) Or even start out with the "free" tier of a year's service on a "micro" instance on Amazon ECC that you get when signing up for the first time.

When it comes to building and running games, hardware and bandwidth are super cheap problems to solve, compared to everything else. Couple that with the significant technical challenge of building a scalable, robust, cheat-proof peer-to-peer system, and it's pretty easy to see why nobody with actual experience has ended up doing it that way.

The reason this idea came up is because i was reading about cloud computing (nothing to do with the $50-$100 a month for a dedicated server...), and the current project i'm working on is an mmo. There are so many computers connected to the internet that are doing nothing but sitting idly, and thinking about that gave me the idea that there are a lot of unused but available resources that could potentially be put to use. I thought if all processing was done on other machines, hacking would be much more difficult because you would have to hack into another persons machine on top of knowing how to hack the actual program. That's just one of the things i thought could be beneficial to this. Maybe your computer is too slow for all the computing necessary to run the game, so having the computations done on other machines would allow you to play the game. Maybe people not using their computer could stay connected for others to use their computer resources. This was all just an idea that came to me, and all i really asked was if this was done before, if so, is there any documentation on it, and if no, what would have to be done to make something like this work. I can see this is not an efficient way to base an entire mmo on, but i still like the idea of being able to use the vast amount of resources available on the net, since a dedicated server can't run the entire game for every client, meaning clients will have to do some sort of processing on their own (i suppose this depends on the requirements of the game though, along with the server). I hope this helps clear up the underlying motive here

I guess to be even more specific, what really got me started was knowing that collision detection is an expensive operation, and accurate results are vital to most games. I don't think most servers are capable of doing collision detection operations for every client, every 1/30 of a second, along with everything else the server or servers have to do to keep the game running. This means the collision detection will have to be done on the client. It's not very hard to make a hack for games to walk through walls, or fly around, or turn walls off. if all this was done on another "random" machine, you wouldn't be able to hack your own game on your computer. of course, hacking whoever is using your computer's resources could be a problem, but this is why i asked for what would need to be done if this WAS to work

I don't think most servers are capable of doing collision detection operations for every client, every 1/30 of a second, along with everything else the server or servers have to do to keep the game running.[/quote]

Why not?

I was thinking rather than full simulation the other clients would mostly be doing validation. You could weight the responsibilities of the clients so slower ones process fewer objects and communicate with fewer people. Given that a client needs to know about all of the players within a certain distance of them, and are executing state prediction on them anyway, all the server would have to do is get a general average of the selected clients' state of the selected object, and send that out to the other clients. If you randomly choose which clients to pull each update from, it makes it harder to coordinate a clan to take over the gamestate, and if a client takes too long to respond the weights could eventually drop it's authority to 0. This would at least reduce the load on the server, though you would still need a central server to store saved gamestate and connect clients to the network.

One might also look into how BitTorrent does things, given the amount of data it pushes through it's P2P network.

if there are hundreds or thousands of players playing an mmo, and the world is really large, if the cd was done on the server, wouldn't that be pretty laggy? players would see their characters rubber banding around i'd think, and the server would need to have every players collision detection environment loaded, and have to be doing collision detections at least every couple frames or so. I just think that's a lot of work for a computer. Of course, it depends on the server or servers hosting the game, but on a regular dedicated server, isn't that too much for it to handle?

firestryke that kinda gave me an idea. Maybe what could happen is that the world is sectioned off into chunks, and every section has a group of people doing the computations for those sections. the central server stores who is computing what sections, so when your character walks into another section, his processing is turned over to the clients doing processing on that area. The weights thing could be taken into consideration too

the world is really large, if the cd was done on the server, wouldn't that be pretty laggy?[/quote]

"the server" is actually physically many CPUs. Just add as many CPUs as you need, assuming you've solved load partitioning and profitability. One 1U chassis with 12 hardware cores for 1,000 simultaneous online players is not a big cost -- and you can probably do 10x that amount without running into lag, at something between 10 and 30 Hz.
Also: MMO games slice their load into many small servers not only for load balancing, but also because it seems to be impossible to design a world economy and game systems that remain relevant to everyone in a 1,000,000 user world -- 10,000 users seems to be the practical limit, and an indie should probably aim for 1,000 per shard.

Also: MMO games slice their load into many small servers not only for load balancing, but also because it seems to be impossible to design a world economy and game systems that remain relevant to everyone in a 1,000,000 user world -- 10,000 users seems to be the practical limit, and an indie should probably aim for 1,000 per shard.

Well, that's more of a design limitation than anything else. EvE being the prime counter-example here where everyone is in the same shard, not same physical server... the idea of physical servers is kind of archaic with modern load balancing techniques, where you can spin up instances of your various proxies, game-play servers, and database processors as needed.

Thanks for those last two replies, i didn't know it was normal for the heavy processing to be done all on the server. I like the idea of being able to use everyone's resources, but i suppose there really is no point when the servers can have enough resources to do it all.