Hi, I like to know ask to setup the network architecture for multiple servers behind a single port that is persistent.

My setup is

1. Mulitple servers running Enet (UDP custom protocol). The connection are persistent.
.e.g Client A connects to Server B, and the load balancer/router need to route everything.

For HTTP, I can do serverA.domain.com, serverB.domain.com, which the load balancer can use host headers to route. But Enet is UDP and a custom protocol.

Is there a software router to be able to do this? 

Yes there should be stuff around to do this. It works off the source address/port number and using those to identify which server machine to route to. (It needs to be both because of outbound natting -- the same internet IP may represent >1 client).

Magic search term will include "routing" "UDP" and "sticky" or "affinity".

There's going to be a timeout setting within which the packets arriving from a given IP/PORT tuple will be sent to the same target as the previous ones from that tuple.

Any hardware load balancer should be able to do UDP DNAT load balancing, at least based on source IP:port.

Many people seem to simply do this with iptables (on Linux) but I think Nginx can do it, too.

Note, by the way, that the player doesn't connect to "serverA.yourgame.com," they connect to "loadbalancer.yourgame.com" which picks one of the servers to forward the player to. If you can give each player a separate name to connect to, you don't need the load balancer; just connect directly to the server.

Thanks for the replies. I just found too that Nginx have UDP load balancing. I'm currently using IPTables but I'm not sure how scalable it can be. I'm looking at thousands of servers.

I do have some specific requirements, I need the clientA to connect specifically to a serverB, so I cannot use the load balancer to decide the routing. When the client connects, the client can make a HTTP request to a backend endpoint, which decides which server the to connect to. I'm hoping to be able to set the routing on the "load balancer" to set the routing. I'm not familiar with hardware routers but I don't think they have API to control it programtically.

Currently my research points to using a software load balancer like Linux Virtual Server. Either that or I have to write my own router but I'm trying to avoid that and use existing solutions.

 

I need the clientA to connect specifically to a serverB, so I cannot use the load balancer to decide the routing.

Then you don't need a load balancer. Give each client a public IP, and let each client connect to the sever in question. Done!

(IIRC, Roblox has something like 12,000 public IP addresses for their servers...)

The other option is to use a different port for each destination server, so the load balancer would route port 2001 to server A, port 2002 to server B, and so forth. However, I don't know how high a packet rate you can get up to in the end in this way -- you'll very likely need multiple load balancers to keep up with "thousands" of servers.

Anyway, public IP per server. Seems absolutely easiest!

Yes,it is not a load balancer but rather a router that can be controlled by API. We don't have enough public IPs in the long run, we might go a lot more than 12K.

I can't use a different port, it has to be same port beause of a certain requirement. I can put out a few public IPs but each IP should be  covering a few hundred servers.

Until IPv6, which seems it will never be implemented.
 

Your requirements seem pretty weird. Why can't you control the port? Why can't you get IP addresses for servers? What game has more than 12,000 separate servers, yet has to live with these requirements?

Anyway, it seems to me that the best thing for you is to put the intended target route into your network packets. The packet header would then contain "please forward me to server instance X." You would write a simple proxy that would receive the UDP packet on the determined port, look at the X value in the header, and forward the packet as appropriate.

Note that there really isn't a better way to do it using "just" plain UDP, because two users may be behind the same NAT firewall, and your system may decide that user A should go to server X and user B should go to server Y, and thus you can't just use the remote IP address as the route determinant. Similarly, some NAT firewalls rewrite the source port, so you won't have a well-defined source port for each user.

Who says it is a game ? Anyway I can't talk about it specifically but I'm looking at a much larger scale than 12K servers so public IP address is definitely not enough. 

I need to control the outgoing port from server because on mobile 4G network, some ISP (tested) block packets if the packets come from other ports.

That's what I'm hoping to avoid doing to write my own since I have too many things to code, and see if there are existing solutions available.

Sounds like you have a pretty custom need that's not "standard" and you probably wouldn't find a good solution for that.

Btw: what do WebRTC browsers do on those 4G networks? Do they not work? Might be worth looking at.

Webrtc would work because the ISP do not block the webrtc ports.  In my setup, if I change the Enet servers to webrtc, I would face the same problem. Imagine each of my Enet server is webrtc, and I do not have enough public IPs, so that's my issue now.

If I server the webrtc over it's non standard ports, likely the ISP would block them too.