I've been currently building a server using C#. I've also been building a test client in Unity. I am familiar with C# and this was the quickest/easiest way for me to get something working. I am currently using a custom message class that is serialized to do communications between the two. While everything does work, I keep rethinking my design, and as of right now, I am contemplating scalability. I'm currently using the .Net TCP classes for the networking.

To start, it's just 1 server running on my laptop (for testing) using No-IP so I can connect from the outside world. Once connected, the server/client can send my custom message class back/forth depending on what each other is doing or wants to do. Then I started thinking (that's never good) that I don't really want to rewrite anything for expandability. I'm not really thinking that the game will go anywhere or get high numbers, so I know that I shouldn't think about this, I just keep coming back to it so I can do it "right" from the start.

The reasoning is because I can reuse the design in other projects. So I started making a sketch of how I think the flow would go with 1 Main/Login server and multiple "spin up servers". The client would initially connect to the main/login server. Upon a successful login, the client would drop that connection and connect to one of the "spin up servers" (identified by the main/login server telling it which one to connect to). Now, all client communication would go to the "spin up server".

But now I rely on the client negotiating connections which can be problematic. But I don't think we can solve this as the client needs to connect to a new IP/Port. Now, the "spin up server" would act like my server which is already coded because the client can only interact with it.

The spin up servers always maintain a connection the main/login server so they can communicate. Thus, we can code in logic to do cross-spin up server communication by sending messages through the main/login server.

So if client 1 on spin up server 1 wanted to send a message to client 2 connected to spin up server 2, the relay would be something like this:

- client 1 -> spin up server 1 (send message to client 2)

- spin up server 1 -> loops all its clients (it can't find client 2)

- spin up server 1 -> main/login server (send message to client 2)

- main/login server -> loops all clients (from all spin up servers) -> finds client 2

- main/login server -> spin up server 2 (send message to client 2)

- spin up server 2 -> client 2 (here is your message)

This just seems abnormally bloated, especially if we start having lots of cross server communication. Is this really the right way to go about multiple servers though? To be fair, I ideally will only start with 1 spin up server. It's just for future proofing and as a concept that can be used on other projects.

Also, might as well mention that the current game idea is a tcg/ccg. So it's not as real time as an MMO or action combat game. Of course, to stay synchronized across all servers, only the main/login server would have access to the database (I think).

At this point I am currently wondering if this is right at all? Are there any suggestions? Warnings? Etc, I'll take any advice on the matter...

P.S. I have read tons of threads on the forums, but I don't recall going into much detail. I have also read this [url="http://dl.ifip.org/db/conf/iwec/icec2006/LimCKS06.pdf"][Dynamic Load Balancing for MMO].

loops all clients

Wouldn't that be much better done with a hash table look-up?

Sounds like a typical application level SDN, where the main server acts as a controller, servers as switches and clients doing host-to-host communication. Check out the following links for more info:

https://en.wikipedia.org/wiki/Software-defined_networking

https://class.coursera.org/cloudnetworking-001

I collected some links about this and you can find them at http://www.gamedevpensieve.com/network/infrastructure . I would try it out with a Login, Main and Game servers.

Login: Only handles logins, create new user, reset mails and all other things related to that. Access user table of database only. When user is logged in it is handed over to main server.

Main: It's main job is to handle game servers and assign players to them. If to many players are on a server split them and move them, if to few merge game servers. When it recive a new player from the login server assign them to a game server.

Game: Almost all the work are done toward the database as it is already good at handling sync and verification of data. Chat messages can be sent by writing them to the database and the server that handles the player currently get changes for that player.

Wouldn't that be much better done with a hash table look-up?

Sure, there are lots of better ways to do look ups than just using a loop on a list per-say. However, the logic is still the same, find the server we need to send to...

Spinningcubes, thanks for those links, they were great!

So it sounds like the game servers should access the database directly.

Chat messages are just an example. But if client 1 writes a message to the database for client 2, how would client 2 know there are new messages? It doesn't seem feasible for client 2 to poll the database constantly looking for these. That means all clients connected would be polling the database for just messages, constantly. I can certainly see the use of the database in the chat example, as then all messages are streamlined, in order, and can be received by the clients in the same order.

I do like the idea of having a separate Login and Main server though. I'm unsure about the "handoff" or if the client would just handle the connection stuff. Being C# I can't serialize the TCP classes, or classes that use the TCP classes.

Sure, there are lots of better ways to do look ups than just using a loop on a list per-say. However, the logic is still the same, find the server we need to send to...

My deeper concern is that, if you believe that "look up a client" is "looping through clients" rather than "a constant time look-up," then you're probably not at the experience level where you will make good distributed systems decisions. Thus, it may be that you're trying to bite off something you aren't really prepared to actually take on yet.

if client 1 writes a message to the database for client 2, how would client 2 know there are new messages

In general, databases are for persistence, and messaging is for updates. Some databases do support "subscribing" to updates, although these often end up being scalability bottlenecks, or single points of failure. Compared MongoDB (source of many WTFs) and Redis Pub/Sub (a little more sane) and ActiveMQ (intended for data streams like real-time stock prices.)

Games typically build their own, because the specific needs of the games is very different from any other application. Typically, you will not round-trip every client update through a database, but instead keep game/simulation state in RAM in persistent process servers, and only update the database when it's important (player trade, significant event, occasional checkpoint, etc.)

So it sounds like the game servers should access the database directly.

Game servers should be responsible for handling game-related tasks, not database-related tasks. They should send persistence events to a separate database server (i.e. "player 1 picked up sword 2"), and the database server should figure out how to persist that information to the actual database.

Chat messages are just an example. But if client 1 writes a message to the database for client 2, how would client 2 know there are new messages? It doesn't seem feasible for client 2 to poll the database constantly looking for these. That means all clients connected would be polling the database for just messages, constantly. I can certainly see the use of the database in the chat example, as then all messages are streamlined, in order, and can be received by the clients in the same order.

Chat functionality can be implemented using separate chat servers that exist orthogonally to the rest of your gameplay servers. You can then create chat channels to parallel game instances, and players are none the wiser.

Offline chat functionality ("e-mail", for all intents and purposes) can be implemented much like a real e-mail setup where the client polls a mail server occasionally for new messages, or maintains a persistent connection if you want/need immediate notification. Again, the key takeaway is that it's a separate server responsible for handling a specific piece of functionality. You're not piggybacking on something else.

I do like the idea of having a separate Login and Main server though. I'm unsure about the "handoff" or if the client would just handle the connection stuff. Being C# I can't serialize the TCP classes, or classes that use the TCP classes.

The hand-off is usually pretty straightforward. The client logs in, and receives its credentials. The login server tells the client to connect to the game server at <ServerIP, ServerPort>, and it also tells the game server to expect a connection from said client with said credentials, within some timeout period.

it also tells the game server to expect a connection from said client with said credentials, within some timeout period.

Typically, you don't even need to tell the game server that. Specifically, you can just issue a login/session token to the client. The token is something like "playerid : time-issued : signature" where "signature" is calculated as HMAC("playerid : time-issued : some-secret")
Your game servers and login servers would both know the secret, and could verify the tokens. When the time-issued is in the future, or way in the past, ignore the cookie.
If it matters which server the player connects to, you'd also include "intended-server" within the session token. Or you could have a different shared secret for each game server.

All the information is great!

It sounds like I will be opting for a Login, Main, Database and Chat constant servers, then a variable number of Game Servers.

The client connects to the Login server which checks authorization within the database and tells the client to connect to the main server. The main server does the "load balancing" by deciding what Game server the client should connect to. The client then connects to the game server.

Now, is it feasible to have the game servers connect to the chat server? So the client could send a message to the game server with chat information and the game server sends that along to the chat server rather than having the client handle the separate game/chat connections.

I'll have to ponder over this and come up with some new designs to lay out.

is it feasible to have the game servers connect to the chat server?

Yes.

Login server which checks authorization within the database and tells the client to connect to the main server

"Login" and "Main" could be the same server software, at least if you buid them as horizontally scalable. If you design each as a SPOF, and use Bcrypt/Scrypt for verification, then keeping login as a separate role makes sense (because that's CPU heavy.)

What is the kind of game you will be running? If it's player-hosted levels/matches, then the role of the "main" server would be more like a matchmaker. If it's a "shared world" then it's more of a coordinator. If it's "MMORPG style with multiple parallel shards" then you should consider whether main is sharded or not, as well as whether chat is sharded or not.