🔒 Locked

A small bundle of UDP questions..

Started by HopeDagger Jan 6, 2006 at 12:26 PM 5 replies 2k views

Original Post

Jan 6, 2006 at 12:26 PM

..or something along those lines. After searching 8 pages back and the fairly unresponsive Google-gamedev.net search gimmick, I caved in and wrote up a new thread. :) With the basework done on my current project, it's finally time to move on to networking the confangled thing. My development language is Delphi, so I don't have the same luxury of having a slew of networking libraries whimsically lined up for me. There are a few, but I've decided to take the high-road and write my own server/client network code with vanilla Winsock. (Or is that the low-road?) And so, I present the curious passerby with a few questions about UDP that I'd like to know before I really get going with my coding adventures. Off we go! :) 1) Are there any speed/performance issues in using send()/recv() after connect()ing as opposed to sendto()/recvfrom()? Or is the different only syntactical? 2) For the server end of a UDP-based program, what method is the most efficient? From the top of my head, there's the select() route with FD_SET's, WSAAsyncSelect() via Win32 messages, blocking with multithreading (which I've heard is not quite wise :P), and I've seen a mentioning of just running a recvfrom() constantly to handle in all packets from all users, at one point. Not sure on that one. I'm just looking for the most speed-effective route for a server that will be handling up to a maximum of ~32 players. 3) Is broadcasting with UDP particularly effective, or is broadcasting only send it over the local network? I'm a bit foggy on how this works. And is this related to multiplexing in some form? 4) On the client end of this UDP-hodgepodge, I need an equally efficient method of getting data from the server, and sending it out. Would a blocking thread be affordable here, since it's only one socket, or is that still unpreferrable compared to the alternatives? What ARE the alternatives in this case? :) 5) I'm aware that when sending data via TCP all of the data is oftentimes not sent; that you need to continuously loop through and ensure that everything is passed over the network. UDP works on datagrams, so does this mean that if you perform a send the entire packet will be send, and no possibility of just a fragment? When receiving, is this also the case? 6) What's a reasonable number of packets for a server to be sending out per second? Heck, for just bouncing player movement (assuming I'm sending 2 per second) that's received to all players, with 20 players, that's 40 sends per second to 19 players. That's 760 packets per second! :-x Is this reasonable, or is there a better way to be handling this? Phew, that should cover the bulk of it. Any advice, ideas, cookies, and the like are all welcome. Thanks! :)

Anon Mike

Jan 6, 2006 at 3:18 PM

1. I'd be surprised if using send on a connected UDP socket would have any significant differences than using sendto. Try it and find out.

2. For low scales like that select is probably sufficient and is certainly the easiest. For highest performance in win32 you want i/o completion ports. My advice is to make this layer of your app easily changeable, go with select, and plug in a completion port version if it becomes necessary. As above try it yourself and see, it's not that hard to cobble together something experimental. The only thing to really avoid is a dedicated thread per client.

3. Typically you can only broadcast on a LAN. Gateways block broadcasts to keep them from getting out of control. And to answer the common next question - no, multicast typically does not work across the Internet.

4. Typically there's only one socket on the client. I usually go with a standard non-blocking socket and just call recv[from] periodically. You can use a dedicated network thread for this if you want but it's likely to be overkill.

5. UDP has no fragmentation / messages concatenation / etc issues at the app layer like TCP does. You either get the whole packet or you get nothing at all. The usual caveats of getting the same packet multiple times, getting packets in the wrong order, getting very late packets, etc, etc apply.

6. 6-10 updates / second / per client is probably reasonable. It depends on how big your packets are and what kind of game you have. An FPS probably wants more updates/sec than a MMORPG for example.

markr

Jan 6, 2006 at 3:39 PM

Quote:
Original post by HopeDagger
1) Are there any speed/performance issues in using send()/recv() after connect()ing as opposed to sendto()/recvfrom()? Or is the different only syntactical?

I'd be extremely surprised if it makes any difference.

In any case, you always want to use recvfrom() to check that the packets came from the expected place, even if there was only one expected place they should have come from.

Quote:

2) For the server end of a UDP-based program, what method is the most efficient?

You only need (minimum) one socket. So the scalability problems with using select with many TCP sockets don't exist for UDP.

Quote:

I'm just looking for the most speed-effective route for a server that will be handling up to a maximum of ~32 players.

Even if you were using TCP and one socket per player, scalability issues would probably not be a problem at that scale.

Quote:

3) Is broadcasting with UDP particularly effective, or is broadcasting only send it over the local network?

It's completely effective and yes, it only works for the local network.

Quote:

I'm a bit foggy on how this works. And is this related to multiplexing in some form?

You're probably confusing broadcasting with multicasting. Multicasting is fairly complicated and not all routers support it. Broadcasting is simple, and almost no routers support it (by design) - it will almost always only broadcast to your LAN.

Quote:

4) On the client end of this UDP-hodgepodge, I need an equally efficient method of getting data from the server, and sending it out. Would a blocking thread be affordable here, since it's only one socket, or is that still unpreferrable compared to the alternatives? What ARE the alternatives in this case? :)

You probably have some kind of game loop running continually. In your game loop, you may as well poll the socket for receiving packets. For sending, you should always send at a rate which doesn't fill the outgoing buffer up anyway (although this is clearly bandwidth-dependent).

There is no point in receiving data more often than you run your logic, even if they arrive more often. The OS may as well buffer them for you.

Quote:

5) I'm aware that when sending data via TCP all of the data is oftentimes not sent;

In TCP all data will be sent (eventually) OR you will receive an error (eventually) and the connection will be terminated. UDP provides no such guarantee. It sends errors occasionally, but that's not an indication that the connection is permanently gone.

UDP error handling is not the same between different operating systems. In particular, Linux handles it significantly differently than most others. I'm not entirely sure whether Windows even ever sends UDP receive errors.

Quote:

UDP works on datagrams, so does this mean that if you perform a send the entire packet will be send, and no possibility of just a fragment?

IP fragmentation happens at the IP level, invisibly to the UDP layer. In the case that a packet is fragmented, it will be reassembled and you'll see either all of it or none of it. This is transparent.

Quote:

6) What's a reasonable number of packets for a server to be sending out per second? Heck, for just bouncing player movement (assuming I'm sending 2 per second) that's received to all players, with 20 players, that's 40 sends per second to 19 players. That's 760 packets per second! :-x Is this reasonable, or is there a better way to be handling this?

I guess it would depend entirely on the game. But you might want to consider sending more than one piece of information per packet. You could also consider only sending pertinent information to a particular client.

It's probably a better idea to coalesce multiple pieces of information into one packet where reasonable. Packets should generally not exceed the path MTU, which is typically 1500 bytes including some layer(s) of headers (not sure which). This limits you to approximately 1k packets, but you'll be limited by bandwidth too.

IP fragmentation will allow you to send larger datagrams, but I'm unsure as to whether this is helpful; it could conceivably be counter-productive (as fragments require headers too).

Good luck.

Mark

HopeDagger

Jan 6, 2006 at 4:07 PM

Awesome. Thanks very much for the replies, both of you (++)! You've managed to clear up pretty much all of my worries and questions. Now I'll toss a few more into the mix, just to finely tune the knowledge I now have. ;)

Quote:
You're probably confusing broadcasting with multicasting. Multicasting is fairly complicated and not all routers support it. Broadcasting is simple, and almost no routers support it (by design) - it will almost always only broadcast to your LAN.

That sounds right -- I have a knack for confusing things with other things. :P So, given this, is there no way to send a packet to several addresses at once over the internet, or am I just dreaming? :)

Quote:
For sending, you should always send at a rate which doesn't fill the outgoing buffer up anyway (although this is clearly bandwidth-dependent).

This brings up another question: when a call to sendto()/send() fails whilst using UDP (non-blocking), is it unwise to jam it into a loop for several more tries until it successfully sends, or is this considered bad design? Is there a way to determine the best time to send data from the server/client via select(), or is that overkill in this case?

Quote:
It's probably a better idea to coalesce multiple pieces of information into one packet where reasonable. Packets should generally not exceed the path MTU, which is typically 1500 bytes including some layer(s) of headers (not sure which). This limits you to approximately 1k packets, but you'll be limited by bandwidth too.

Yes! When I read that (the second time) it finally popped into my head. By having the server send updates twice a second at that constant rate instead of sending the data to ALL clients each time it receives data, I can save monumental amounts of packet-sendings. A huge thanks for helping me realize this. :D

Thanks again, the both of you. :)

markr

Jan 6, 2006 at 4:30 PM

Quote:
Original post by HopeDagger
given this, is there no way to send a packet to several addresses at once over the internet, or am I just dreaming? :)

You can use multicasting.

Or rather, you can use multicasting provided:

- You can figure out how to (it isn't easy)
- The client and all the servers support it
- All intermediate routers support it

The last two, these days, should be fairly likely. The first one is trickier.

To do multicasting, each router along the path must know that it's a multicast packet and where it's going, to "fan out" the packets appropriately.

Of course, you can always fallback on just sending several packets to different destinations with the same (or similar) data. A lot of games do that.

You'll always need a backup plan isntead of multicasting in case some of your clients don't support it.

NB: I have never written a program which uses multicasting, this is based on what I've read.

Quote:

This brings up another question: when a call to sendto()/send() fails whilst using UDP (non-blocking), is it unwise to jam it into a loop for several more tries until it successfully sends

Very unwise, this is busy-waiting.

Busy-waiting is considered bad form, as it locks the machine into a loop doing, well, nothing.

You should either retry a little bit later, buffering the messages in the application, or just ignore the error (after all, UDP is unreliable anyway, so presumably your protocol can cope with lost packets).

Quote:

, or is this considered bad design?

It's considered bad design.

It's not bad to go of and do some other useful stuff that you were going to do anyway, and then retry a bit later. It's bad to sit in a tight little loop trying to stuff more packets into the pipe like mad :)

Quote:

Is there a way to determine the best time to send data from the server/client via select(), or is that overkill in this case?

Probably overkill, I guess.

If the server frequently gets EWOULDBLOCK from sendto(), then it may be that there is a problem with either that client, or it simply doesn't have enough outgoing bandwidth. In any case, even if handled correctly, your users will get a very bad game experience.

Mark

hplus0603

Jan 6, 2006 at 5:18 PM

Quote:
Original post by HopeDagger
1) Are there any speed/performance issues in using send()/recv() after connect()ing as opposed to sendto()/recvfrom()? Or is the different only syntactical?

It's probably less efficient to use send()/recv() with connect(), because you'd need one socket per remote machine. Typically, you'll only use a single UDP socket for all your communication needs in a program, and use sendto()/recvfrom().

Quote:
Original post by HopeDagger
2) For the server end of a UDP-based program, what method is the most efficient? From the top of my head, there's the select() route with FD_SET's,

You're going to be using a single UDP socket. How you poll/read that is mostly unimportant -- you can make it nonblocking and try draining it/writing it each time through your main loop, if you want. Threading will introduce locking, which may introduce more costs than it gains you, especially when only serving 32 players.

Quote:
Original post by HopeDagger
3) Is broadcasting with UDP particularly effective, or is broadcasting only send it over the local network? I'm a bit foggy on how this works. And is this related to multiplexing in some form?

Broadcasting is great if you can use it, because it's a single datagram on the network, getting to all the listening peers on the local subnet. Broadcasts are not routed, though, so they are only visible inside the closest router.

Quote:
Original post by HopeDagger
4) On the client end of this UDP-hodgepodge, I need an equally efficient method of getting data from the server, and sending it out. Would a blocking thread be

See previous answer.

Quote:
Original post by HopeDagger
5) I'm aware that when sending data via TCP all of the data is oftentimes not sent; that you need to continuously loop through and ensure that everything is passed over the network.

You're talking about really big buffers passed to send(), when the available TCP window is small, and the kernel buffer is almost full? This is pretty rare, but as you say, can still happen.

Quote:
Original post by HopeDagger
UDP works on datagrams, so does this mean that if you perform a send the entire packet will be send, and no possibility of just a fragment? When receiving, is this also the case?

All or none of the data you pass to send() will be accepted by the kernel/network stack. If it needs to fragment and reassemble during transport, that is transparent to the sending and receiving peers.

Quote:
Original post by HopeDagger
6) What's a reasonable number of packets for a server to be sending out per second? Heck, for just bouncing player movement (assuming I'm sending 2 per second) that's received to all players, with 20 players, that's 40 sends per second to 19 players. That's 760 packets per second! :-x Is this reasonable, or is there a better way to be handling this?

Coalesce updates! When you send a packet to a single player, send updates for all the state that's changed since the last time you sent to that player (or, perhaps, since the last packet you sent and have gotten an acknowledge for).

Other than that, do the math on packet overhead (28 bytes per UDP datagram) and actual payload; divide your server upstream bandwidth by that number, and allow at least 20% for spikes, management and noise.

HopeDagger

Jan 6, 2006 at 11:00 PM

Quote:
Original post by hplus0603
Quote:
Original post by HopeDagger
6) What's a reasonable number of packets for a server to be sending out per second? Heck, for just bouncing player movement (assuming I'm sending 2 per second) that's received to all players, with 20 players, that's 40 sends per second to 19 players. That's 760 packets per second! :-x Is this reasonable, or is there a better way to be handling this?

Coalesce updates! When you send a packet to a single player, send updates for all the state that's changed since the last time you sent to that player (or, perhaps, since the last packet you sent and have gotten an acknowledge for).

Aye, aye. I didn't do this for my last online game, and I went through lag hell when I was hosting a 7-player game. Everyone was playing the game at least 5 seconds behind. ;) I'm definitely going to be taking advantage of this!

Quote:
Other than that, do the math on packet overhead (28 bytes per UDP datagram) and actual payload; divide your server upstream bandwidth by that number, and allow at least 20% for spikes, management and noise.

Is there any easy way to determine your connection's download/upload rate? [EDIT: Found this. :) According to it, I'll have no problem hosting 32 players. Huzzah.]

Thanks for the additional answers! :)

(Out of all of the forums, I treasure my time in the Networking realm the most. So many folks with awesome answers :D)

[Edited by - HopeDagger on January 6, 2006 11:00:44 PM]

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.