🔒 Locked

[C#.NET Socket]BeginSend behaves unexpected.

Started by Annoyed Jun 4, 2009 at 5:29 AM 18 replies 7.7k views

Original Post

Jun 4, 2009 at 5:29 AM

I am trying to create a net library for a multiplayergame. My chosen implementation is to use asyncrhonous socket communication. In my session(s) I use a fixed size send and receive buffer, pinning them to memory in order to marshal/unmarshal structs. Thereby I have also said that my messages are structs. And I do mean plural, since there is a struct per message type. Example: -Chat -Position update -State transitions -etc etc Needless to say that the messages vary in size. In order to be able to handle every type I need a large enough buffer(s). When a struct is marshalled into the buffer i also know the size in bytes and then use the Socket.BeginSend(byte[],int,int ...) call to transmit the marshalled bytes. The start index is always 0 (zero) while length is the size of the struct in bytes. This goes well until I try to send multible successive packets. Lets say I try to send a chat, transition request and then a chat again. 3 successive BeginSend calls. What I have found out via tracing is that the first message is sent and received correctly. The second is also sent correctly but along with it is the third message truncated to fill out the send buffer. Third send then contains the tail of the thrid message. In other words, if buffer has size 256 and all messages have size 200. Then first send would transmit message 1 (200 bytes), second send would transmit message 2 and 56 bytes of message 3 (256 bytes). Then thrid send would contain the remaining 146 bytes of message 3. Result is that unmarshalling 3rd message on receiver side returns jibberish. I have found a workaround and that is to send the entire buffer (256 bytes) each call regardless of message size. I would like to believe that I don't have to. Such that a BeginSend call with a specified length only sends that much bytes. Is this a bug or a feature as they say? :) Or is my approach unappropriate?

Cygon

Jun 4, 2009 at 5:55 AM

TCP Sockets, whether accessed through .NET or C/C++, whether on Windows or Linux, act like streams: they can split and combine data in any way they want when transmitting to the other end of the socket, as long as the order of the data remains unchanged.

Normally, there's a threshold value, which means the socket will wait for up to some arbitrary time (like 1 ms) for you to put additional data into its buffer before sending. This is done to avoid sending lots of small packets (remember that each packet has a certain overhead, so transmitting 10 bytes with 40 bytes packet headers would be inefficient). This splitting and combining can also happen outside of your control as the packets are being routed and forwarded through various systems on their way if you use the internet.

If you need your message to stay intact, either invent some bracketing scheme (eg. prefix any structure you send with its length) or use an UDP Socket. UDP will transmit only whole packets. However, delivery is not guaranteed, so you'll have to verify arrival yourself.

Annoyed

Jun 4, 2009 at 6:06 AM

Thanks for the reply.
If you are referring to the Nagel algorithm I have disabled it in order to achieve the initially intended behavior.
As in, one packet -> one transmission.
This is assuming that the API call does what it says it does.

I still don't understand why this happens. My own created buffer is just to define the byte sequence to send the underlying system is unaware of its size.
If I send the entire buffer everything works as supposed to byt sending a fraction messes it up (only on successive calls).
The underlying Socket buffer remains at it's default size (8192 bytes).
http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.sendbuffersize.aspx
Which in theory would mean that all messages of 200 bytes would be buffered and then sent.

Jansic

Jun 4, 2009 at 6:45 AM

Quote:
Original post by Annoyed
As in, one packet -> one transmission.

Even if you mange to do this over TCP at the _sending_ end you still need to realise that because TCP is a stream protocol (as Cygon noted) and you can't make the same assumption at the _receiving_ end. There are no messages, no packets, no arbitary boundaries related to 'send' calls, just data in the order it was transmitted. It works if you fix your data to 256 bytes because both ends now know that valid data only arrives on 256 byte boundaries.

Jansic.

Annoyed

Jun 4, 2009 at 7:02 AM

Thanks you just made me realize that the problem might be the receiver BeginReceive call where you also have to specify buffer and length to receive.
http://msdn.microsoft.com/en-us/library/dxkwh6zw.aspx

Since I don't know initially what I am receiving I have the specify the full buffer length.

The confusion remains why it happens after the second message and not after the first message ... It has to remain unresolved :)

So the conclusion is that I have to narrow down the buffer as much as possible to reduce "wasted" bandwith. Possibly setup several channel links.

My current implementation has 512 byte chat messages (+header) and some messages are below 100 bytes total. So sending the entire buffer every time inevitable leads to some excess... Back to the thinking box ...

hplus0603

Jun 4, 2009 at 12:51 PM

TCP sockets work like files. TCP sockets are streams, not packetized. If you want to parse separate packets, you have to first send a length field, followed by the actual data. When you receive, you have to decode the length field, and if there isn't enough data yet, keep the data in the buffer until more comes in.

The merging of packets can happen at any time, depending on network and system conditions.

For more information, see the Forum FAQ.

Sirisian

Jun 4, 2009 at 1:31 PM

Also to make this painfully obvious I'll give you an example.

Client sends "foobar\0" to server and then sends "barbaz\0". Server calls recv and it could get 1 byte or 10 bytes. You don't know until you check the size. So you have to write this data to a buffer as you get it.

The server could do this:
call recv and get a buffer of 1 byte: "f"
call recv and get a buffer of 2 bytes: "oo"

The reason I'm saying it could be just 1 byte is because I've seen people write an unsigned integer or a short to represent the length and do things like:

call recv get buffer of 1 byte and the programmers accidently pulls off 4 bytes to grab the length. Oh no. Kaboom.

So make a buffer that you use to process incoming packets per player. The naive way of doing this is having:
int packetLength = 0;
List incomingBuffer = new List();
and then in your recv callback

while((int size = recv(buffer)) != 0){  while(!buffer.EOF ... or something like that)  {    if(packetLength == 0)    {      while(incomingBuffer.Length != 4 && !buffer.EOF)      {        //Write a byte to the incoming buffer      }      if(incomingBuffer.Length == 4)      {        packetLength = read the first 4 bytes of incomingBuffer      }    }    if(packetLength != 0 && incomingBuffer.Length != packetLength)    {      for(read in bytes until buffer.EOF or incomingBuffer.Length == packetLength)      {        // Read a byte from buffer and append it to incomingBuffer      }    }    if(packetLength != 0 && incomingBuffer.Length == packetLength)    {      // In case you like to process packets in your main thread I put a lock      Lock(packetQueueMutex)      {        packetQueue.Insert(incomingBuffer);        packetLength = 0;        incomingBuffer = new List<byte>();      }    }  } }

That's like the super naive version but works for small games.

Annoyed

Jun 4, 2009 at 3:59 PM

@hplus0603
Yea the image has sunk in from previous posts and after trying to relate this more precisely to the FAQ.

@Sirisian & hplus0603
What I do is the have the first field of the struct as a byte which constitutes the type of the struct being transmitted.
Thus by peeking the first byte in the buffer I know how many bytes are supposed to follow and I can unmarshall.
This seemed so plausible to my initial assumption. further I'd like to say that it worked flawless until I tried sending successive packets/structs (within a loop or similar). But I suppose it was coincidental.

To make it work in all cases I just have to pass identical fixed length parameters to the BeginSend and BeginReceive calls on the respective sides and this will lead to excessive bytes being transmitted. Then to reevaluate if it still seems plausible :)

All in all I guess in order to keep the varying size packets being marhalled transmitted and then unmarshalled is another argument to use UDP in stead.
I need to stop being lazy ...

Thanks both

Sirisian

Jun 5, 2009 at 7:33 AM

Quote:
Original post by Annoyed
What I do is the have the first field of the struct as a byte which constitutes the type of the struct being transmitted.
Thus by peeking the first byte in the buffer I know how many bytes are supposed to follow and I can unmarshall.
This seemed so plausible to my initial assumption. further I'd like to say that it worked flawless until I tried sending successive packets/structs (within a loop or similar). But I suppose it was coincidental.

Sounds like you just have a simple bug. Also I find it nice to create my own "packets" so they act as atomic state updates. Since you chose to use structs they might look like:

2 bytes length of packet1 byte struct typeN byte struct1 byte struct typeN byte struct1 byte struct typeN byte struct2 bytes length of packet1 byte struct typeN byte struct

The reason being that sometimes you want to update the full-state of the game lets say with one packet (A single transaction). You might not want to evaluate it over multiple recv calls where the state can be changing between time steps.

Quote:
Original post by Annoyed
All in all I guess in order to keep the varying size packets being marhalled transmitted and then unmarshalled is another argument to use UDP in stead.

You can handle the same thing with TCP as I just mentioned. I also gave you the pseudo code for receiving "packets" with a length prefix. When you get a full "packet" you can then process it at that point.

Annoyed

Jun 5, 2009 at 8:40 AM

Yes the bug is that I don't consider the possibility of receiving partial packets/structs.
Also I use a fixed buffer in order to use the Marshal tool in the .NET framework. In order to do so you need to pin the byte buffer to memory using the GCHandle.

byte[] buffer = new byte[ size ];GCHandle handle = GCHandle.Alloc( this.buffer, GCHandleType.Pinned );IntPtr ptr = handle.AddrOfPinnedObject();

The only downside is that you have now told the garbage collector to ignore that buffer and you have to call Free() when done.

Now you can just marshal and unmarshal the struct to and from the IntPtr.
In my case:

static class MessagePacker{    internal static int Pack( object package, IntPtr bufferPtr )    {        Marshal.StructureToPtr( package, bufferPtr, true );        return Marshal.SizeOf( package );    }    internal static T UnPack<T>( IntPtr bufferPtr ) where T : struct    {        return ( T )Marshal.PtrToStructure( bufferPtr, typeof( T ) );    }}

This only works if the message is packet into the buffer from index 0 (zero).
So when I receive partial packets also the whole thing gets out of sync.
If i pack a message from the sender and then flush the entire buffer it works. but it isn't pretty and lots of padded bytes being transmitted.

So the fixed buffer thing has to be reevaluated if i want to use the TCP :)

hplus0603

Jun 5, 2009 at 12:36 PM

Quote:
To make it work in all cases I just have to pass identical fixed length parameters to the BeginSend and BeginReceive calls on the respective sides and this will lead to excessive bytes being transmitted

That won't work. Even if you pass 20 to BeginSend, and 20 to BeginReceive, at the other end, BeginReceive may end up only receiving 10 bytes, and the next BeginReceive might get 5 -- or might get 50, if there is more data pending. You have to treat a TCP stream as you would a file on disk, with the additional gnarl that the "receive" operation may return less than you asked for, even when the file isn't at the end.

Washu

Jun 5, 2009 at 12:43 PM

As hplus stated. The best thing to do is ensure that each packet has an associated header giving the appropriate details about the packet, such as its length and type. When receiving you should buffer the data you have obtained until you have at least a full packet, which can then be dispatched to the appropriate handler.

Also, you should probably write a custom marshaling setup, using what you've demonstrated is not flexible and also has some rather nasty behavior under certain circumstances.

Annoyed

Jun 5, 2009 at 12:57 PM

That may be, but in my case it does work coincidental or not.
But in any case I got the picture that I have to treat it as a stream :)

Annoyed

Jun 5, 2009 at 1:24 PM

Quote:

The best thing to do is ensure that each packet has an associated header giving the appropriate details about the packet

As I am using structs and using the sequential structure layout. I set the first element of the struct to be a byte value that is unique per package type. Therefore by peeking the first byte i know what type of struct and also the size of it.
Not saying that it is the best solution.

Quote:

... using what you've demonstrated is not flexible and also has some rather nasty behavior under certain circumstances.

Care to elaborate?

Washu

Jun 5, 2009 at 1:46 PM

Quote:
Original post by Annoyed
Quote:

The best thing to do is ensure that each packet has an associated header giving the appropriate details about the packet

As I am using structs and using the sequential structure layout. I set the first element of the struct to be a byte value that is unique per package type. Therefore by peeking the first byte i know what type of struct and also the size of it.
Not saying that it is the best solution.

Pray tell: What do you do for chat messages? Send a fixed length buffer every time?

Quote:

... using what you've demonstrated is not flexible and also has some rather nasty behavior under certain circumstances.

Care to elaborate?[/quote]

struct Stuff {  string Text {get;set;}}

Care to guess what this marshals to? I'll give you a hint question: Is string a value type or a reference type?

Annoyed

Jun 5, 2009 at 2:06 PM

Quote:

Pray tell: What do you do for chat messages? Send a fixed length buffer every time?

yes, knowing that it is not optimal (at all).

Quote:
struct Stuff {  string Text {get;set;}}
Care to guess what this marshals to?

Nothing, there are no fields in your struct. But if there was a text field I would specify it as:

[StructLayout( LayoutKind.Sequential )]struct Packet{    [MarshalAs( UnmanagedType.ByValTStr, SizeConst = 256 )]    private string text;    public Stuff(string text)    {        this.text = text;    }}

Now it is marshalled as 256 char array.

Quote:

I'll give you a hint question: Is string a value type or a reference type?

It is a reference type. Your point?

Washu

Jun 5, 2009 at 2:12 PM

Quote:
Original post by Annoyed
Quote:

Pray tell: What do you do for chat messages? Send a fixed length buffer every time?

yes, knowing that it is not optimal (at all).
Quote:
struct Stuff {  string Text {get;set;}}
Care to guess what this marshals to?
Nothing, there are no fields in your struct. But if there was a text field I would specify it as:

Wrong, it does have a field. And it's a reference, marshals to sizeof(int*) relatively meaningless bytes.

Quote:

*** Source Snippet Removed ***
Now it is marshalled as 256 char array.

Indeed, and now you've got a problem in that you have no idea how many of those bytes you actually need. And for chat messages, that's damn important.

Annoyed

Jun 5, 2009 at 2:26 PM

Ok I don't want to split hairs over what your uncompilable struct marshals into :)
And the importance you are referring to i presume is wasted bandwidth?
Because I don't need to know how many of the bytes I need because when I unmarshal it contains string as it was before it was marshaled.
But as I stated above I am aware it is not optimal.

Washu

Jun 5, 2009 at 2:41 PM

Quote:
Original post by Annoyed
Ok I don't want to split hairs over what your uncompilable struct marshals into :)

My "uncompilable" struct compiles quite fine if you're using the latest version of C#. [grin]

Quote:

And the importance you are referring to i presume is wasted bandwidth?
Because I don't need to know how many of the bytes I need because when I unmarshal it contains string as it was before it was marshaled.
But as I stated above I am aware it is not optimal.

True, it's a minor issue, and you have to remember to make sure you properly annotate all of your packet structures. Not to mention, a custom marshaller (actually, it would be a serializer), tends to be a lot more flexible in the data format of the packets, allowing for things like bitwise compression.

Annoyed

Jun 5, 2009 at 3:05 PM

Quote:

My "uncompilable" struct compiles quite fine if you're using the latest version of C#. [grin]

I stand corrected >.<
Never saw that one before, gawd it saves tons of redundant coding.

Quote:
Not to mention, a custom marshaller (actually, it would be a serializer), tends to be a lot more flexible in the data format of the packets, allowing for things like bitwise compression.

Yea that would be the best solution I agree. But again I'm just being lazy at picking the idea that happens more or less by it self.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.