Original Post
Hey, Im just currently doing a learning excercise into MMO servers and im currently coming up against a few problems, which i could easily solve, but it would feel like ive solved them in a cheap and not so extendible way... so i just wanted anyones opinions... Currently i have the following nodes: - Gateway node This is an externally facing node which connects up with X amount of login nodes to verify the connected users then once they login the gateway should pass them over to a realm node. - Login node This is a simple node that hooks up to the DB directly and validates any username/password requests. - Realm node This is basically a node per game world, so it would store all the current connections to a given world and deal with updating players and passing on updates to the world, its effectivly a per realm gateway. - World node These are created dynamically by the Realm node to process requests etc, this way if a character requests a movement thought the realm node, it would pass it on to an available World node that would validate its movement then send back the movement to take to the realm to forward on, they would also do any other in-game calculations such as damage etc. - Memcached Although not a node as such there are a few layers of caching, such as a global cache that any world node can access, so this would pull back items/areas that have already been loaded. Then there would be per realm caches to store the current players to save constantly hitting the DB, although snapshots are frequently taken of players to update the DB. Now thats just the basic outline im going with at the moment, and all the problems really stem from it being distributed and dynamic, so new nodes can be created as and when they are needed, here are the current problems ive got... - X amount of Gateway nodes The client has to know about the gateway nodes, as these are the first things they will hit to start accessing the world. Although going forward if you had 100,000 people wanting to connect 1 gateway wouldnt cut it, so i would need to deploy more. However how would the client know about them unless there was some kind of hardcoded configuration file with the client saying there are X gateway servers at XXX.XXX.XXX.XXXX. Only way round this i could think of would be to have a single gateway manager, that would take a connection then pass it on to sub gateway nodes, this way it would not hold the connection and would just pass it straight through, whereas the current gateway nodes hold the connections until either the client times out or logs in. - Having internal nodes knowing about other internal nodes Similar to the problem above, when a user logs in they are then asked which realm they wish to play on (akin to WoW realm choice), then once they click it the gateway node needs to find out: A) The specific realm nodes address B) If it is online C) If it is accepting new players Now currently if you have 10 realms you would expect 10 active realm nodes, but unless the gateway has a config file telling him where they are it wouldnt know how to tick the A requirement. I thought of maybe having a tiny DB that could be checked for online realm nodes, but it feels like a cheap hack, but it would allow for quick changeover if a realm node went down and a new one needed to be put up. As i dont really want to hardcode IP addresses within other nodes as the whole point of it being dynamic and distributed is that it should be able to expand and contract as and when is needed. Once it has an IP it can easily tick B/C requirements by just talking to the server. This issue also comes up when looking at how dynamic world nodes would be created, as its easy enough for me to get the Realm node to spawn some new world threads, but this would mean its always going to be on the same box... so its not going to offload any of the calculations, so the other boxes would need to have an app listening for spawning requests or something, again needing a config file or DB or something... As im sure you can see im still trying to get my head around the best sort of way to distribute the system as they would be on seperate boxes and even possibly seperate networks. So its not like i could do an IPX style network spam to see if there are any listening nodes as it wouldnt work with remote nodes, the DB idea sounds the best to me so far but it feels like a way round the problem opposed to acctually solving it, its a case of how should dynamically created nodes know where they are supposed to be feeding their information to. Unless you just pre create them and give them all configs, but then they are no longer dynamic... If anyone has any advice on the subject it would be great as ive been trying to find some decent articles online but not found much yet...