Making a multiplayer game with physics feature

There are Gaffer's articles. He is a member of these boards.

Quote:

Play Motorama

Awesome game BTW. I remember that game for a few years ago. I think it was called something else though.

[Edited by - oliii on January 21, 2007 6:30:19 AM]

yup, i read gaffer's article awhile back. Even he insists that you need to have a deterministic physics engine/system otherwise your game will requires heavy sync.

Well then, I don't know for sure. I've yet to see a game that implemented networked physics completely accurately, which as you said, would require deterministic behaviour, and some pretty constraining requirements (lock-step). There was an article about that on Gamasutra, netowrk engine for X-Wing, or Tie-Fighter (one of those). Can;t remember the article exactly.

Most of the time, they just do with heavy sync. Usually, dynamic objects are treated like any other object, and their update sent (consumes bandwidth). So, the sync is not perfect, but without deterministic physics, not much choice. That would have been great though, as we also started doing replays on the basis that the physics engine was deterministic, but that was just not an option. It's very hard to make a 100% deterministic physics engine. So now, we have to record game states, which makes the replays much bigger than they could have been (say 1 meg instead of 1k).

In the game I am working on, we solved the problem by not having any dynamic objects :) Just barriers, and things you can destroy and blow up. The dynamic objects we use are only for cosmetic reasons, a la counterstrike-source. We'll think of proper networked physics 'for the next time'.

Quote:

Original post by yanuart yup, i read gaffer's article awhile back. Even he insists that you need to have a deterministic physics engine/system otherwise your game will requires heavy sync.

No he doesn't - he comments:

"comment #1, i should clarify that for the techniques in this article, you dont require exact determinism, just that you need to get it as deterministic as possible. i should clarify this in the article — its confusing!

Comment by Glenn Fiedler — October 20, 2006 @ 12:02 pm"

Whether or not ODE (as you're using it now) is deterministic enough - you can probably only establish by actually using it.

Even if it's not apparantly deterministic you might/should be able to make it so - for example if it uses a random numbers you should be able to seed the generator the same on the different machines. Google for ode and deterministic.

I heard something a while back about using integer/fixed point numbers for deterministic physics engines. I guess if you used 64 bits (which is becoming native anyway) then you'd have enough accuracy for game physics without FP errors. The biggest part of that is deciding what level of magnitude is accurate enough :)

The thing with deterministic engines is that the math resolution should not matter. If you take two numbers, and apply a math routine to them, they should return the same result on the same machine at any time. For that, there are FPU overrides to set the math precision. But discrepancies can arise if you have two different processors (say, AMD and Intel, or even two different processors from the same manufacturer). As soon as a deviation occurs (even one bit), it's game over. The simulations will diverge, and pretty quickly.

So yeah, in that case, having a fixed point math could potentially remove the discrepancies. But it's hard work. There are so many things to look after, especially if you use multi-threading.

Hmm.. okay, so what I concluded so far is : fully deterministic physics engine is as elusive as bigfoot and you have to work your way around it.
I guess that's okay, I really like the input based approach rather than heavy update on the client cause it's bandwith friendly :) but it sure damages your input response time and in a game where your input is very crucial in the game I don't think that'll work.

I'm trying to make a hybrid version by taking the best of two worlds, any experience on this?
Maybe I should get a book or something :)

Your best bet is to figure out how to compare physics states simply. Maybe some kinda function that, say, adds all distances squared and compares them across the network? You need to establish that the objects on both sides are *roughly* equal, and work out which one is server if they need to resync.

Talking from FPS game experience, the input lag is solved by running the player physics concurrently on the server and client.

client
------
1) Take player commands
2) run the player physics with those commands
3) Package inputs + player physics state + sequence number into command packet
4) Bundle the last 2, 3 packets into a message (duplication to overcome packet loss), and send message to the server.
5) bis repetita.

server
------
1) for each player, wait for command packets to arrive.
2) queue the packets in sequence order.
3) run the player physics with those commands.
4) Compare your simulation results with the results found by the client.
5) If they diverge, send a correction packet to the client to reset the player state.

so, it's kinda client-side physics, but the server has an override option. The player will not run physics on the server until he received the player commands. This leads to player hanging in mid-air on the server (and other clients), if the connection drops, and weird things like that.

The diff check can be as tight as you want, because as soon as you diverge, then it will be game over pretty quickly anyway. As said, the simulation will run exactly the same 99.9% of the time, putting aside difference in FPUs. The correction will be needed when you bump into another player (which is impossible to predict accurately on a client).

As for dynamic objects, Dead reckoning can be a solution. But it's innacurate, obviously. You'll have to run into lock step, and act on inputs. Then fake the observable result on screen using interpolators.

BTW, it's all explained in Gapher's article. As he also says, predicting dynamic objects that do not have a clear ownership is hard, and I agree with him that the 'client side prediction' feels more like a hack than a proper way of doing things, and will become obsolete.

We tried networked physics, but it's just too much of a hassle, and not enough time, and it looks rubbish tbh (lots of latency issue, wobble, ect...). So all our physics objects are static, and are more for gameplay interraction (exploding barrels!), and shaping the levels (barriers). All the dynamic objects (bottles, cardboard boxes), are just there for eye candy, and don't really interract with player (they just brake apart when you walk on them).

This makes it a lot simpler, but it would be nice to have true dynamic objects. But when you start considering latency, irregular updates, peer to peer model (have to on consoles), and the huge bandwidth requirements for even one single simple dynamic object, it's just too much and not worth the hassle.

Next time, we are gearing towards incorporating proper physics objects into the world. However, it will require a common effort, from the physics guys, and from us, the network monkeys. It's not gonna be pretty, but it's all for the cause!

[Edited by - oliii on January 24, 2007 3:13:36 AM]

I've done my research playing online racing games and it seems that the best way to do it is to let client has its own authority in physics (for the object it's controlling but not for other proxy objects). The server will receive the inputs message and client physics state, if there aren't any big difference between the client and server physics state then the server actually corrects itself according to the client msg.

If the difference is big enough to raise suspicion of cheating or massive lag then the server will kicks in its authoritive function to send the correction. Otherwise the server will follow client's simulation.

This will ensure that each client will play at the most responsive control available cause some games (like racing) require that.

The only thing left is to avoid players using trainers or some sort.. pheww.. It's a hard task making multiplayer games, isn't it ? :)

Quote:

ODE is actually a non-deterministic engine

This is not true. As long as the inputs given to the simulation are the same, the output is the same. It's getting the inputs to be the same that is the headache.

Quote:

I've yet to see a game that implemented networked physics completely accurately, which as you said, would require deterministic behaviour, and some pretty constraining requirements (lock-step)

See one here. (The link is to the updater app which will download the necessary files ~3MB) As you mention, it is lock-step so its not a quick-reflex style of game.

Quote:

But discrepancies can arise if you have two different processors (say, AMD and Intel, or even two different processors from the same manufacturer). As soon as a deviation occurs (even one bit), it's game over.

I have not found this to be an issue, as long as the correct settings are applied, and that is the headache. The server for the above app is running on Linux, clients are Windows. All cpus must be x86/87 and implement the IEEE floating point standard properly, which pretty much all do.

Quote:

Your best bet is to figure out how to compare physics states simply.

If your simulation should be the same for all participants, you can just compare CRCs of their world states, so that even one bit of difference will show up. I used a method where each object in the game world gathers the variable which relate to physics, makes a CRC. You can extend this to get a CRC for the world as a whole, and compare these. I also realised that I then needed to find out which part of the object was different. For this I made a text file dump of the world state with floating point numbers represented like:

string floatToHex(float f){	char tmp[32];	sprintf(tmp, "%08X", *((int*)(&f)));	return string(tmp);}

Then I used subversion's diff feature to see where the dump files differed. That diff tool is really handy because it will even tell you which characters differ within one line.

Quote:

It's a hard task making multiplayer games, isn't it ? :)

You can say that again.

Quote:

Quote: I've yet to see a game that implemented networked physics completely accurately, which as you said, would require deterministic behaviour, and some pretty constraining requirements (lock-step) See one here. (The link is to the updater app which will download the necessary files ~3MB) As you mention, it is lock-step so its not a quick-reflex style of game.

Hey, that's pretty cool!

Quote:

Quote: But discrepancies can arise if you have two different processors (say, AMD and Intel, or even two different processors from the same manufacturer). As soon as a deviation occurs (even one bit), it's game over. I have not found this to be an issue, as long as the correct settings are applied, and that is the headache. The server for the above app is running on Linux, clients are Windows. All cpus must be x86/87 and implement the IEEE floating point standard properly, which pretty much all do.

I've seen it being a problem, but that was ages ago (around 7 years ago), when working on a PC game. It was only minor (well, a non-issue really, just something we noticed), because we used the determinism only for replays. And it was a bastard getting it deterministic, and especially the debugging.

Quote:

Quote: It's a hard task making multiplayer games, isn't it ? :) You can say that again.

Yep. But it's the future. Now consoles are gearing more and more towards the online play, these sort of problems will get solved eventually.