P2P matchmaking frustration

Started by
6 comments, last by hplus0603 4 years, 4 months ago

Hi people, so I once made a simple multiplayer game, but the architecture was pure client-server. Now I'm trying my hands on P2P using a central server for matchmaking only, but I hit a brick wall here.

So here's what the server does:

  1. Start udp socket, listening at [any] interface on port 3232
  2. Whenever client connects:
  3. Add this [client] to [list of connected clients]
  4. Send this [client] the following info:
  5. his public ip+port (as seen from the server)
  6. all his peers (connected clients so far)'s public ip+port
  7. And every 200 ms, do the following things to connected clients:
  8. keep sending him ping message, and receive pong message back (to keep the line open and measure latency)

And for the client, it's also simple:

  1. start udp socket, connect to server
  2. receive all kinds of message from server
  3. when receiving a list of peers, do this:
  4. send a hello message to every peer's public ip+port
  5. keep doing this every 200ms

Thing is, the packet to peer's public ip+port never arrives at the other end. I even tested this @ localhost, still no luck, wtf. I started the Server, then the clients, all communications between server to clients works really well. But client to client packet just doesn't get through. I also tested it using netcat (try to send garbage to the client's ip+port as seen from server), still nothing gets passed. I log the client's receive function heavily, but still only packets from server gets received. What did I do wrong? I thought that's how UDP Hole punch works?

I understand that not all kinds of network connection supports UDP hole punch, and I'm using 4G Broadband USB Modem at home. But the thing is Torrenting just works! so I thought the UDP hole punch should work. Even if it can't work because of my network, it must work @ localhost, amirite?

Advertisement

EDIT: It works @ localhost, the client directly send message to each other (using supplied ip+port from server's point of view). But it still doesn't work over the internet. I read again, and it could be made to work if somehow Client A sends a packet to Client B's public ip+port, but somehow modify it so the packet's source seems to be sent from server's address. In other words, spoof the source address. But alas, since I'm using java, there seems to be no viable way to do that, sigh.

Some NAT firewalls don't like it when they receive an incoming packet before they've sent an outgoing packet, and thus will attempt to put such sources on a blocklist or otherwise prevent the NAT introduction from working.

There are two ways around this:

  1. Coordinate the NAT sending, such that client A will send messages to client B and client B will send to client A at more or less the exact same time (meaning, when they receive a synchronizing message from the server.)
  2. Start sending with a short TTL (say, 2 hops to start) and then increase it 1 hop at a time until you reach connection. You may want to send these packets at a quicker cadence than 200 ms, though, to avoid spending too long waiting for connection. Once you have connection, you can increase the TTL to the max again, to be robust to changes in internet pathing.

If these don't work, then you will need to figure out some way to get packet dumps from the inside AND OUTSIDE of each firewall involved in the transaction.

enum Bool { True, False, FileNotFound };
hplus0603 said:
Coordinate the NAT sending, such that client A will send messages to client B and client B will send to client A at more or less the exact same time (meaning, when they receive a synchronizing message from the server.)

The peers send packet to each other pretty much almost at an instant, as seen here (they start sending dummy message as soon as they get 1 or more peer addresses):


hplus0603 said:
Start sending with a short TTL (say, 2 hops to start) and then increase it 1 hop at a time until you reach connection. You may want to send these packets at a quicker cadence than 200 ms, though, to avoid spending too long waiting for connection. Once you have connection, you can increase the TTL to the max again, to be robust to changes in internet pathing.


Alas, i'm using java (it's for android), and so there's no way to set socket TTL for UDP socket. Your method looks like network path discovery (traceroute)? the thing is even tracert/traceroute doesn't work on my machine (cause of the network I guess, because it works just fine using my office's network).

But as I read here , looks like the NAT implemented by 3/4G carriers are just kind of impossible to be punched. Thing is, I use bittorrent just fine. And they're P2P. How did they do it? Also, since I kinda hit a roadblock here, do you know any P2P client library for java? I looked at RakNet but it's C/C++ only. *sigh*.

Anyway, here's the p2pfail video.




[quote]pretty much almost at an instant[/quote]

is that "actually instantly, in response to packets sent the same time to both clients from the server?" Then it's instant enough. But the timing needs to be quite tight, because the point is to make sure that point A doesn't see a message from point B before they have sent a message to point A, and vice versa -- the first message must be crossing in the aether!

[quote]there's no way to set socket TTL for UDP socket[/quote]

That's probably not actually true. Worst case, you could write some JNI to open a socket, configure a socket, send/receive bytes on a socket, and close the socket -- it's a very narrow interface, with very basic datatypes.

But it looks like the socket implementation has this already; you can use Reflection or something to get out the SocketImpl and then call setOption() on it, passing the appropriate IPPROTOIP/IPTTL values.

[quote]NAT implemented by 3/4G carriers are just kind of impossible to be punched[/quote]

If that's actually true, then you need to either provision a server, or see whether you can make IPv6 work in your favor.

enum Bool { True, False, FileNotFound };
hplus0603 said:

[quote]pretty much almost at an instant[/quote]

is that "actually instantly, in response to packets sent the same time to both clients from the server?" Then it's instant enough. But the timing needs to be quite tight, because the point is to make sure that point A doesn't see a message from point B before they have sent a message to point A, and vice versa -- the first message must be crossing in the aether!

[quote]there's no way to set socket TTL for UDP socket[/quote]

That's probably not actually true. Worst case, you could write some JNI to open a socket, configure a socket, send/receive bytes on a socket, and close the socket -- it's a very narrow interface, with very basic datatypes.

But it looks like the socket implementation has this already; you can use Reflection or something to get out the SocketImpl and then call setOption() on it, passing the appropriate IPPROTOIP/IPTTL values.

[quote]NAT implemented by 3/4G carriers are just kind of impossible to be punched[/quote]

If that's actually true, then you need to either provision a server, or see whether you can make IPv6 work in your favor.

Thanks for the response. I've been researching it quite heavily the last weekend. What I found are:

  1. either use JNI to call some library that does TTL setting underneath
  2. make sure one of the peers was not behind a symmetric NAT
  3. the setOption does nothing on connectionless socket. Even when it's used for UDP in Java, only applicable to a special class for that called MulticastSocket which is quite inappropriate unless I'm targetting LAN only p2p.

I've tried my p2p app above, and it all works in almost all cases except when both client are behind a symmetric NAT. I tried port guessing but the thing with symmetric NAT is the port assignment is unpredictable (most of the time). It's funny, in my office, the port assignment was totally random, while in my home network, although it also uses symmetric NAT, the port assignment is incremental.

Come to think of it, all of my friends who play PS4 over the network were all using public IP or is behind a UPnP compatible NAT. None were using broadband heh. But I guess that's it for my journey, for now I'll stick to pure client server. Perhaps when IPv6 is common here I'll reconsider it.

Yeah, symmetric NAT is a pain, and, according to many experts, actually breaks protocols like UDP. (It kind of depends on how you squit when you read the specification.)

Generally, people will fall back to a central server that forwards traffic in these cases.

enum Bool { True, False, FileNotFound };

This topic is closed to new replies.

Advertisement