Snapshot Interpolation

OK so just to update on my current situation, to solve the problem of a new message coming before the interpolation finished and to fix the jittering what I did was to create a queue, now that obviously creates issues the more messages arrive the longer the queue and the longer the delay, also dependent on how much of a delay one adds. I am creating a pong game so one could say that it's a reflex game and @Kylotan in his original post mentions that this technique wouldn't work on those.

Is there a way to do snapshot interpolation for reflex games?

You might want to look at the Entity Position Interpolation Code library:

https://github.com/jwatte/EPIC

We used a type of extrapolation for a game we created. It works well on slower games and masks latency effectively: http://www.kinematicsoup.com/news/2017/5/30/multiplayerprediction

Thank you both! I added both urls to my read list and already started going through the first. I'm not sure extrapolation is the right way for a pong game because of the skill involved if it gets corrected it might look like the other player isn't making up his mind.

It's not clear why you didn't try the simplest solution to the original problem: when a new future position comes in from the server, you lerp directly towards that from wherever the client representation is now. In other words, the currently lerped position becomes the 'now, T=0' position and the received position becomes the 'later, T=N' position, where N is some number of milliseconds.

The reasoning for using a time in the future is because of the following:

• if you instantly snap to whatever position the server reports then it's obviously going to look very jerky - so you want to smooth that over some period of time. (Let's call that a 'time buffer' for the purposes of this post.) You interpolate between a start position and a next position.
• that time buffer effectively adds more latency, because your client has that extra delay before it moves the entity where it needs to be. So you want the buffer to be as short as possible to reduce this.
• On the other hand, you always want to have a 'next position' queued up in the future or the entity will stop moving while it waits for the next snapshot to come in. (If you think about it, the 'instant snap' situation can be considered a special case where the interpolation is over 0 milliseconds.)
• So, you choose a time buffer length that is short enough to not add too much extra latency, but long enough that you always have a next position from the server queued up, even in the face of varying message ping/latency (known as 'jitter')

Regarding Pong - I think this method would be fine. You have very few entities to update and therefore very low bandwidth requirements, so I'd just suggest sending messages as frequently as possible.

@Kylotan I did try to "override" the current position but still was getting jittering maybe if I had stopped and optimized how much into the future it should jump rather than changing to a queuing system would hide the jittering. I might give it another try to see if I did a mistake at the time.

as you say the extra latency started creating issues even if it was set to 50ms. I currently solved that by speeding up the lerp but that also doesn't create a very smooth result but a passable one. I realize that I l get better results the more I play with the numbers but shouldn't there be a result that makes it look like it's running 30 fps? Currently I'm sending a packet every 2 frames.

If you were seeing discontinuities in the position, then you weren't treating the currently interpolated position as the new start position, or you weren't resetting the interpolated time at that point.

If you were seeing the object continually stop and start, then your updates were arriving too late to ensure that there was always a future position queued up.

One good way to debug this sort of issue is to slow everything down - drop the network send rate to something like once per second, set your update rate at something like 20 or 30 frames per second, scale up all other time values accordingly, and do a lot of logging to see when things change in ways you don't expect. Get it working correctly at the slower speeds, where it's easier to debug any issues. Then ramp up the speed to what you actually need.

Also, ensure you understand how to interpolate properly. If your interpolation looks like Lerp(start, end, 0.5) (or any other constant) then that is not truly interpolating linearly between the values over time.

If this isn't enough information to help you, then you will probably have to show your snapshot handling and interpolation code.

It might've been a case of a bug in the server code as well which didn't send packets in the same order they were sent. I added an indexing method when I did the queuing to drop "previous" packets.

OK so my jittering in the end wasn't due to mistakes in snapshot interpolation code rather some extra physics were interacting with my object when they shouldn't thus network code was setting the object where it should be and the physics engine tried to move it elsewhere, it could also be what I mentioned above about indexing my packets but mostly the physics caused the issue.

To comment on @Kylotan 's previous messages, the reason I asked why we add the future was because it felt unatural to manually add extra latency but of course there isn't any other way to interpolate. I think what would make your answer more complete would be to increase the speed of interpolation by something small so that it eventually catches up to the extra latency that was setup, basically that was what confused me as I thought for skill games 50ms might create edge cases but when I fixed the physics issue I saw that it won't.

8 hours ago, Tipotas688 said:

I think what would make your answer more complete would be to increase the speed of interpolation by something small so that it eventually catches up to the extra latency that was setup

No, because that's not necessary, nor does it make sense. If the object is moving, then your local interpolated data will continually be following those positions, N milliseconds in arrears, where N is the length of the buffer you chose. If the object is not moving, then your local version will catch up to the final position within N milliseconds. There is no need to change the speed of anything.

I don't see why it makes no sense. It can catch up earlier. Since you know where you want to be, it gives you an extra variable to adjust and extra accuracy which again you can adjust.

The point of this sort of system is that a state update takes some length of time (say, 'M') to reach the client from the server, and then some length of time (say, 'N') before the client fully reflects that change, to help ensure the client has received another state update from the server before the current one is reached.

You can adjust N slightly on a per-message basis to account for fluctuations in M. In other words, if a message arrives sooner than expected, you might want to take longer to interpolate towards it, and vice versa. This keeps movement smooth.

It is reasonable to consider decreasing N during play so that it's not too long and imposing unnecessary client-side latency. This will be determined by M and your send rate.
It's also reasonable consider increasing N during play so that it's not too short and leaving entities stuttering around, paused in the time gaps between reaching their previous snapshot position and receiving the next snapshot from the server. This is determined by the variance of M (jitter) and your send rate.

Often N is left at a fixed value (perhaps set in the configuration), picked by developers as a tradeoff between the amount of network jitter they expect to contend with, and the degree of responsiveness the players need to have. And if you're happy occasionally extrapolating instead of just interpolating, i.e. you are happy to take less accuracy for less latency, then you can reduce N down even further.

The idea to "catch up earlier" doesn't make sense in isolation. Earlier than what? The idea is that you always allow yourself a certain amount of time to interpolate smoothly towards the next snapshot because you still want to be interpolating when the subsequent one comes in. You don't want to be decreasing that delay over time because of the stuttering problem above, unless you have sufficient information to be able to do so.

@Kylotan took me some time to reply to your last message as we were really busy with the release of our new game but now I can get back to this and give it another look.

What you're saying is correct, maybe I didn't phrase it properly to explain what my issue is. The example I've picked for the above in case I haven't mentioned was making a pong game which needs to be pretty accurate.

I've made the system to be what you suggested but even with the variable N depending on lag but because of the ~50ms of delay the "rackets" in a lot of cases don't reach in time basically breaking the gameplay. My question is mostly whether snapshot interpolation can actually be used in skill games which accuracy is important or should I be looking into something more complex?

It's not entirely clear what you're asking. Let's strip it back to the basics: every networked game where there are entities with non-deterministic movement (e.g. a player-controlled object that changes direction when the player chooses) is going to have to send a stream of updates about its current state ("snapshots"). Those updates will come in less frequently than the visual frames are displayed, and therefore you have a choice of no interpolation at all, meaning the object moves immediately, but in a jerky and disjointed manner, or you apply interpolation between those snapshots, meaning it moves in what appears to be a smoother manner but which must be done over a period of time following the inbound message. That's pretty much a fact of life, whether you need 'accuracy' or not, whether it's a skill game or not.

What you then get to choose, are a bunch of tradeoffs about how you apply this stuff, and it's basically "low accuracy, low latency" at one end of the continuum and "high accuracy, high latency" at the other end of the continuum. There is no magic "high accuracy, low latency" approach because information necessarily takes time to travel. You can reduce the latency by using extrapolation instead of just interpolation, but the very nature of extrapolation means it will sometimes make incorrect predictions, and that you've traded away some accuracy. Sometimes that is the right choice for your game.

Sure I'm with you, I think I'm reaching the same conclusion that extrapolation is required. The test game I'm making ( pong ) is sending 30 packets a second, basically every frame to make sure I get the best result possible. If I lose a packet I use the next one, if there's a delay I snap to it.

What I notice is that a user that has 100ms delay plays with issues that are game breaking, a user that has 60ms plays much better. Being a pong game it needs accuracy to play ok and as you say there are trade-offs but I'm currently not seeing great results even though I have such low latencies. Makes me wonder whether a game like pong can be done with just snapshot interpolation or whether it also needs extrapolations as well

On 1/10/2018 at 2:43 AM, hplus0603 said:

You might want to look at the Entity Position Interpolation Code library:

https://github.com/jwatte/EPIC

This help to me

20 minutes ago, gainapple said:

This help to me

Sure, I have read it but since I'm new to network coding I want to make sure I grasp some of the main concepts before I add more to my plate

3 hours ago, Tipotas688 said:

Sure I'm with you, I think I'm reaching the same conclusion that extrapolation is required. The test game I'm making ( pong ) is sending 30 packets a second, basically every frame to make sure I get the best result possible. If I lose a packet I use the next one, if there's a delay I snap to it.

What I notice is that a user that has 100ms delay plays with issues that are game breaking, a user that has 60ms plays much better. Being a pong game it needs accuracy to play ok and as you say there are trade-offs but I'm currently not seeing great results even though I have such low latencies. Makes me wonder whether a game like pong can be done with just snapshot interpolation or whether it also needs extrapolations as well

There are a lot of unknowns which can affect how your game works.

1. You've not posted any code that proves you're sending at 30Hz. It's very easy with modern libraries to call something 30 times a second but for it to only be actually sending something once a second or after 100ms or similar. That adds latency.
2. Similarly, even if data is being sent 30 times a second, we've not seen the code that guarantees it is being read at a similar rate. It is common for networking libraries to buffer up input and only poll it occasionally.
3. Even if somehow it was possible that your reads and writes are hitting the wire instantly, the server processing loop also tends to add latency. If the server runs at 30Hz then that's an additional 0 to 33ms per message to relay it to someone else.
4. 100ms and 60ms are not really "low latencies", generally speaking. It depends on whether you're talking about LAN or WAN, and whether this is the round trip time or the direct time, and whether this is measured at the application level or the network level. It's useful to take these different measurements to see where the time is being spent.
5. Pong is a game where the paddles might move the entire height of the screen in a second. If it takes 100ms to get the message to the other player then they're going to see the paddle perhaps a whole tenth of the screen distance away from where the the controlling player sees it. If you add even just 1 frame of interpolation onto that then you're looking at pushing that error up from 10% to 13 or 14%. However...
6. ...usually this doesn't matter. The goal is almost never to have two clients entirely synchronised (because it's impossible), but to make the game playable. The game, in this case, usually only has 1 paddle where the position really matters at any given time. You can use extrapolation on the server to see that a player would 'probably' hit the ball even if it doesn't exactly reach on the server, for example.

That is some great info there! Thanks a bunch! I had the above in mind but someone verifying them is actually great. I'll be re-reading this answer a few times as I'm working out the details on the kinks of server to master server to client and back. Fingers crossed I get better understanding on where bottlenecks happen so I have more educated answers to my questions.