Nice snippet from Carmack talking about the plan for QuakeWorld networking / prediction system:
https://raw.githubusercontent.com/ESWAT/john-carmack-plan-archive/master/by_day/johnc_plan_19960802.txt

@irreversible

irreversible said:
Without local simulation you'll likely have to deal with lag

I want to clarify something, just in case, my issue is with player seeing other players, not with the main player.

Yes, sadly that's something I willn't be able to fix :/, I was thinking to use some tick-rate system, rn server runs logic 100ms and client runs it at 33.3ms, so my idea is to create snap of position every 100ms server & client, and assing a snapId to each one.

Server will send each snapshot it take every 100ms with the current snapId (which is always +1 than the last one), so client will start simulate other players, and will also save snapshots at same rate, so I can compare snapshot 1 from client vs 1 from server, if they don't match (ofc they willn't I will add some threshold like 0.5m), and when some difference between client & server are more than 0.5m client will discard all snaps and continue simulation from last valid point.

Rn I don't care much about packet size and how much data I send, I just want the system to work smooth as possible, then I will deal with compression and stuff like that.

Also, keep in mind, I'm testing in a 250ms environment xD

hplus0603 said:

Working C++ code for interpolation can be found on an old page of mine:

https://www.mindcontrol.org/~hplus/epic/

I tried to implement that system, just to test it, only issue I found is, I don't understand how to get the time from packet, since server timestamp willn't be the same as client one. In that case u always get it from the local GetRealTime() which make everything much easier xD

Kylotan said:

A basic starting point would be:

• track movement in ticks (e.g. a 10th of a second, or a 20th of a second) rather than real-time, so that you have a shared ‘unit’ that all clients and servers can use. I recommend starting with a very low value - e.g. 4x per second, 250ms - so that it's easier to debug and understand. You can even just printf position values to the screen while testing. Then just crank up the frequency later, once you know everything works.
• send an update for each tick rather than just on/off inputs, to avoid network delays producing different results. You can pack multiple updates into one network message to save bandwidth, but beware of the extra latency this incurs.
• updates can include not just the input state but also the locally-predicted position
• have the server apply the movement inputs and compare the outputs to locally-predicted positions. This can help with debugging potential desync situations.
• broadcast the updates out to clients, who buffer them up and use interpolation to blend between them (and extrapolation if a message is missing or delayed). Linear interpolation is a good start during debugging as it is visually obvious what data you are working with. For release you'll usually want something more advanced (e.g. hplus0603's EPIC code, linked in the previous message)
• How you do the buffering and interpolation can vary - shooters will usually try to maintain some sort of carefully-synchronised clock but RPGs can often get away with simply dequeuing the updates for an entity once there are enough to meaningfully interpolate from. It's game-specific.
• you can expect some minor rounding errors, so a tick in which a player stops moving is an opportunity to set a precise stopping position and effectively resync everything.

Thanks! I'm doing some of those tips, I will continue looking how I can make the system.

One issue I found, I fell that the movement is too slow, but if I make it faster, like LERP_SPEED * dt for the interpolation, sometime it's just soo fast, player reach to T = 1 and stop right there.

for question: each client will recv move instruction in different time.
you could use tick in server, says 16 tick per second. then agent could only start move/stop/cast in tick point, and each agent will recv message like [tick 132, forward move, start pos x, y, z]. more optimized, each agent recv a message [tick 132], after by [agent 1, forward move, start pos x, y, z], [agent 3, back move, start pos x, y, z] etc, you should promise message send in order.

since you are make mmorpg(not like room game), you are unlikely to make sure every client in right state now, so do not fire agent action from client, just queue them to server, and fire action from server, that would be much easy

I recommend you make a video describing and showing the problem.

From your initial post, i suspect one problem is you need to send game time with the move messages. You also need to do something to establish (approximate) synchronized game time across cliebts and server. Then you will send a mive command like “client-a started moving at dir from position at TIME”.. that way, all clients can canculate about where they should be now in a way that should match across clients.

Of course if you just instantly move players to the calculated position, they would snap. Ao instead you always lerp between their old location and new estimated position over some number of frames.

cheng1 said:
since you are make mmorpg(not like room game), you are unlikely to make sure every client in right state now, so do not fire agent action from client, just queue them to server, and fire action from server, that would be much easy

Problem with that it's delay, if I queue actions to server, ping will make everything slower.

jeskeca said:
From your initial post, i suspect one problem is you need to send game time with the move messages. You also need to do something to establish (approximate) synchronized game time across cliebts and server. Then you will send a mive command like “client-a started moving at dir from position at TIME”.. that way, all clients can canculate about where they should be now in a way that should match across clients.

Hmm, AFAIK u can't have a sync clock, is just not possible, maybe I'm missing something, do I need to sync game time across client & server, how? I thought sync clock was nearly impossible, if we even can't have atomic clocks in sync xD

It is very easy to make a synchronized game time clock. The synchronized game time is not precise it is approximate. You can use NTP or something even simpler. The idea is to give the server some idea of when an action was started in a high ping or variable ping scenerio.

For example, a spell is broken down into a “pre-cast” animation and an effect animation. If the precast is 0.5s, and cast of 0.5-2s then you start casting and tell the server the game time when you started casting, and play the “pre-cast” animstion in a loop. The server receives the “cast request” and computes the game time the actually spell cast should end, giving the client a grace period of 150-400ms to start the precast in the past, and sends it back to the client.

As long as ping is <400ms round trip, the client gets the “ack” before the 0.5s precast ends, and continues on to the normal cast aniamtion with no percieved latency. This is how a 1.5s spell always feels like it takes 1.5s (as long as latency is < the grace period) even though it is server mediated.

Only if latency is longer than the grace period does the client “feel” it during the cast.. and they feel it as an extended precast animation. If the server never responds, then the client is stuck in the precast animation until it retries or disconnects. You see this exact behavior in wow if someone has a lag spike during the precast animation.

The same type of trick cam be done for movement. The user presses forward, the client starts moving locally, and tells the server the position, game time, and direction it started moving. The server gives the client a grace period in the past to begin movement. If it starts withing the grace period, the server starts the player moving in the past, if not it clamps to the grace period, in both cases it sends the client back a position and game time. If the client get a response too far away from where it is, then it has to “snap back” the player.

Movement can also be done without game time, by using a movement speed and grace period to bound movement updates. In this case the server receives a new position from the client, and as long as it is within the distance a player can move inside the grace period, it accepts the movement command, otherwise it clamps it to the max movement distance and sends the client a snapback correction.

jeskeca said:
It is very easy to make a synchronized game time clock. The synchronized game time is not precise it is approximate. You can use NTP or something even simpler. The idea is to give the server some idea of when an action was started in a high ping or variable ping scenerio.

For what I understand NTP use UDP, which will not work for me, since I'm using TCP.

jeskeca said:
For example, a spell is broken down into a “pre-cast” animation and an effect animation. If the precast is 0.5s, and cast of 0.5-2s then you start casting and tell the server the game time when you started casting, and play the “pre-cast” animstion in a loop. The server receives the “cast request” and computes the game time the actually spell cast should end, giving the client a grace period of 150-400ms to start the precast in the past, and sends it back to the client.

Hmmm, sounds good, I will try to implement something like, but as u said for movement, it's the only system with issues at the moment.

You can use any channel (tcp or udp) for time sync. Just read the NTP algorithm description, it is very simple. That said, using a TCP channel can cause unpredictable delays from packet loss retransmits and other details of the TCP reliability design, so it is better to do time sync over UDP.

You can also get similar latency hiding without time sync.. you just want the client to have a loopable pre-cast anim, and don't trigger the final spell until you get a resppnse from the server.

As for movement, the best way for us to understand what os going on is to see a video or diagram of the problems you are seeing.