Network input handling

I would say that the input handling code is different -- read from keyboard, vs read from network -- but the simulation code is the same. The simulation code just needs to know what actions/triggers are active each time step, not where they came from.

I would assign a small integer ID to each action, so that talking about actions over the network would be cheaper than sending a string. If you have less than 30 actions and they are all binary (on/off) you can even get good efficiency by just sending all actions as a bitmask, assigning them each an id from 0 through (N-1) and using the smallest integer that will fit all the actions. The client and the server then have to agree on the XML file that maps action names to integers, or that mapping has to be dynamically established during session open. (Much easier to make sure they just use the same XML file :-)

So you're suggesting I separate the client and server HandleInput methods? I dunno, somehow, having to separate them doesn't sit well with me. I feel like it shouldn't be too hard to abstract the input state enough so that it the handling code can be the same for both client and server.

As for assigning an ID to the actions, I suppose that will work, but I won't know how many actions there will be, since mods can add new ones. I'll have to make it load all the network related ones from the server, since any changes to the mods can change the ID mapping.

I would suggest that there is no problem here.
Firstly, let's unify the term "event". Each client may handle input with different keys mapped to each event. Therefore, it is best to have an event defined independently of the keys that triggered it. The client and server can use this definition of Event for all simulation systems. We therefore need to manage where the event comes from. This is simple. Both client and server user classes call a "get_input" event. For the client, this polls the keyboard and parses it into an Event class instance, whereas the server does the same but parsing from the network packet.

This seems akin to your suggestion. In the case of Hplus's suggestion, that would make sense for I believe he means the same as you and I have suggested.

Right now I have an InputState class which is constructed each frame with the current/previous mouse/keyboard state by the input manager and then sent out to the various systems. The systems can call IsHeld or IsTriggered to find out about the current state of an action (or event as you call it). I'm starting to think I should make an interface and then have 2 different InputState classes, one that is constructed from the mouse/keyboard state and another that is constructed from the net state. The only issue is that this InputState class also provides access to the raw input state for things that can't really be described in terms of the actions I have set up. Like rotating the screen based on the relative mouse movement. Any ideas for this?

I don't think I say so explicitly in the article, but one of the motivations for this piece was actually to solve this exact problem.

If you follow the general architecture I described there, rigging up different "sources" of input becomes trivial. You can source from keyboards, mice, joysticks, network messages, telepathy... you name it.

So you're suggesting I separate the client and server HandleInput methods?

That depends on what they do! If they handle input type tasks, such as "read keyboard for input," then yes, that's the right seam.
If they actually processes simulation based on inputs that are read somewhere else, then no, that's the part that should be the same!
So I think we're saying the same thing. Reading keyboard vs receiving data from network is different. Updating simulation based on "cooked" input data (events) should be the same.

The HandleInput methods in my classes do indeed process simulation based on inputs read elsewhere. I've made some progress with this. I took the idea of ranges from Apoch's code so I could abstract the mouse X/Y/WheelDelta. I was able to remove public access to the input once I did that.

Now it's just a matter of finding a way to send the relevant actions to the server so it can construct an InputState based off that. Right now my current thought on this is to create a kind of RegisterAction method in the input manager where various systems can inform the input manager about the actions they'll be looking for and, the important part, whether or not its a network related action or not. Then I can just have the input manager check the state of the actions and send out a packet containing any that have changed state. Hopefully that'll work out

Ok, that seems to be working pretty well. The input is being automatically packed up and sent to the server. Next step is to actually feed it to the various systems, but that should be easy. The one thing I was wondering about though is if I should limit the rate input is sent to the server. Its not much of an issue with keyboard input but moving the mouse around (to rotate your character) will pretty much result in a constant stream of input messages being sent as fast as the input can be read. Is that something I should bother with or is it ok to leave it as is?

Rate limiting network traffic is always a good idea. For instance, you could have the client accumulate transmitted input a fixed 30 times a second and send that instead of spamming the server at whatever framerate the game happens to hit.

The one thing I was wondering about though is if I should limit the rate input is sent to the server.[/quote]

You should absolutely run simulation at a fixed time step size, which should be the same on all clients and servers. (This means simulation stepping should be de-coupled from rendering.) Inputs should generally be time-stamped for the particular simulation step number (starting with 0 at beginning of game) they are intended for.

The corollary is then that it doesn't make sense to have more than one input state per simulation step. Additionally, you can trade off latency for packet overhead by packing N steps of inputs into a single packet. You can also compensate for occasional lost packets by pre-emptively including "older" inputs, too, in successive packets. For most kinds of inputs, these will either be very small, or at least RLE encode or compress very well.

So I should handle the input immediately on the local client and then queue up the commands and send them out to the server every 33ms? And then the server will execute them ... how? As soon as it gets them or what?

So I should handle the input immediately on the local client and then queue up the commands and send them out to the server every 33ms? And then the server will execute them ... how? As soon as it gets them or what?

At that point, your specific approach to consistency and lag hiding comes into play. There are a few possible options here, and which one you choose depends on what your trade-off between control lag and consistency and bandwidth is.

I'm kind of stuck with how to handle the input server side. I'm using BEPU for my physics engine in a minecraft type game. This article says the the way to handle client side prediction and server reconciliation is that you tag each input packet with a number when you send it out and then store it locally. The server processes it and sends back the results with that number included. When the client receives it, it sets the world state to what the server says, and then re-applies any further inputs that occurred after that point.

There's 2 problems with that though. First is that I can't just tell BEPU "hey, can you run the simulation for THIS ONE PLAYER". Second is that, even if I could simulate for a single entity, what about all the physics objects you could collide with that would change the resulting position? The valve network article also seems to be tailored for a game with 0 physics: "Then the server moves all other players - only players - back to where they were at the command execution time. The user command is executed and the hit is detected correctly. After the user command has been processed, the players revert to their original positions." Suppose you have a game with blocks falling that could intercept the shot. If you don't rewind them, the results are wrong.

I know getting all the clients synced up perfectly is pretty much impossible and I'm not looking for a perfect solution, and probably not even the best solution, but the valve article seems a bit dated and designed for a static game and the other article suggests a method that seems impossible to implement with my physics engine. I get the feeling minecraft itself doesn't do any of this stuff (I don't actually know if that's the case or not) and still manages to do a half decent job, so if you could give me some advice on what I *can* implement in this situation, I'd appreciate it.

Easiest is to only send the inputs to the server, and only run physics when you get back "here's all inputs for all clients at step X" to the client. This can be fully consistent, but will introduce some control lag. You can work to reduce this lag through UI and display tricks, as well as making sure you turn on TCP_NODELAY (or use UDP) and run at high simulation step rates, but not fully remove it.

I suggest taking hplus's route and trying to do latency masking for each client. Speaking from experience, distributed consistent physics is not easy.

the main issue is lag. Our goal is to have 100+ player servers. I wouldnt want to go much higher since that would cause some serious requirements but a 100 player server would be ideal ( assuming you have the hardware to handle it )

From our previous network attempts we noticed that there was some serious delay on actions that were shown from client to client. Player A could break a block and player b wouldnt see it for 10 min later. While player b would modify a block and player a would see it miliseconds after it was done. I know, in part, this is due to our code but it also has much to do with our choices for how we are doing things. Since that time we have streamlined most of those issues out but we are really stuck when it comes to the physics and other information. Thanks for the help with this guys.

Since that time we have streamlined most of those issues out but we are really stuck when it comes to the physics and other information. Thanks for the help with this guys. [/quote]

If the difference in "seen time" is higher than the difference in round-trip latency, you're probably doing something wrong.
If the difference in round-trip latency is above, say, 300 ms, you're also probably doing something wrong.
Well-implemented networked games take networking into account in physics, input, rendering, and game design, which is why retro-fitting networking into an existing game can be hard.

All Points Bulletin does > 100 players per server using Unreal Engine as a Free to Play game.
Planetside did 500 players per server 8 years ago. (Planetside 2 is out not, but I haven't had a chance to check it out yet)
100 players per server is not at all an impossible goal to hit, assuming you build on top of solid principles and code. Sometimes, that's a bit of a stretch assumption, though :-)

Since that time we have streamlined most of those issues out but we are really stuck when it comes to the physics and other information. Thanks for the help with this guys.

If the difference in "seen time" is higher than the difference in round-trip latency, you're probably doing something wrong.
If the difference in round-trip latency is above, say, 300 ms, you're also probably doing something wrong.
Well-implemented networked games take networking into account in physics, input, rendering, and game design, which is why retro-fitting networking into an existing game can be hard.

All Points Bulletin does > 100 players per server using Unreal Engine as a Free to Play game.
Planetside did 500 players per server 8 years ago. (Planetside 2 is out not, but I haven't had a chance to check it out yet)
100 players per server is not at all an impossible goal to hit, assuming you build on top of solid principles and code. Sometimes, that's a bit of a stretch assumption, though :-)
[/quote]

Planetside 2 does reasonably well with 500+ players in a single region. I will admit there is some serious issues with network stuff but they are attempting to simulate some crazy stuff that we wouldnt even begin to attempt to manage.

I agree that with sound principles the 100 player mark would not be hard to reach... hence why we came here! We appreciate your help and your thoughts as they have helped us to get as far as we have thus far. I will have telanor and goss post up anymore information or status reports so we can see what else can be improved. Again thank you.

The world updates aren't that bad anymore. When we originally had it working, the server was sending out updates as fast as possible. But since I changed it to run at a tickrate of 66, clients are now seeing the block updates before you can even hear the other person say they placed it in vent.

We're having an issue with handling the player input on the server side though. The server typically ends up applying the player's input for a shorter duration than the client does. I can't really see what would be causing it. I'll describe how things are set up right now and maybe someone else can point out the issue. The client examines the input state every frame and checks to see if its different from the last state sent to the server, if it is, it sends the changes. Also, every frame it applies the input and runs the physics update, which is set to run at a fixed rate of 60fps. On the server side, the server runs at a tickrate of 66, which is done by sleeping for however long it calculates it needs to until the next tick. On each tick, it reads all the input that has accumulated, merges it into a single state for each player, applies it, and then runs the physics (which again has its internal update rate set to 60fps).