-SOLVED- TCP Message data not all coming through

TCP is a stream. You could get any number of bytes from the recv call. If you send 100 bytes there's nothing to say you can't get 10 bytes after calling recv. For a packet of your size this is more common. So call recv and keep building your packet with multiple recv calls.

Quote:

Original post by Sirisian TCP is a stream. You could get any number of bytes from the recv call. If you send 100 bytes there's nothing to say you can't get 10 bytes after calling recv. For a packet of your size this is more common. So call recv and keep building your packet with multiple recv calls.

Sorry. I should have emphasized that I already do this. The message sometimes will come in parts, a couple thousand bytes each. When it doesn't work, it's because the number of bytes that it all adds up to is 2 bytes short.

Also after further testing, I've discovered it is not at all consistent as I thought it was. About half the time I get all of the data. The other half I am 2 bytes short.

"the number of bytes that it all adds up to is 2 bytes short."

Is there any chance this is due to some sort of pointer alignment problem? {eg; you're reading into an array of ints, but the message can be truncated on byte boundaries}

Can you write a small test case which sends test patterns through. Then you can send "0123456789" repeats and see whether the bytes are missing from the start/middle/end by checking the results?

If you put a "sleep()" call in your program about where you put the breakpoint, do you also get the full data? If so, this is a timing related thing.

How do you know the data is coming up short -- is it possible it's the check that has the wrong logic rather than the data actually being short?

Does it *always* fail without the pause? Network problems (dropped packets etc) are random(ish) -- if the transmission always fails the same way, then there's logic wrong somewhere.

Quote:

Original post by Katie "the number of bytes that it all adds up to is 2 bytes short." Is there any chance this is due to some sort of pointer alignment problem? {eg; you're reading into an array of ints, but the message can be truncated on byte boundaries} Can you write a small test case which sends test patterns through. Then you can send "0123456789" repeats and see whether the bytes are missing from the start/middle/end by checking the results? If you put a "sleep()" call in your program about where you put the breakpoint, do you also get the full data? If so, this is a timing related thing. How do you know the data is coming up short -- is it possible it's the check that has the wrong logic rather than the data actually being short? Does it *always* fail without the pause? Network problems (dropped packets etc) are random(ish) -- if the transmission always fails the same way, then there's logic wrong somewhere.

Since I'm sending 6783 and getting 6781, I don't believe this could be a pointer alignment problem. This is particularly true of ints, since 6780 and 6784 are divisible by 4, and the numbers are between those two. Perhaps if one of the numbers were divisible by 4... but I'm taking the data into a char buffer, using the standard recv() call, so even if it were a possibility with the number of bytes, I don't think that is the problem.

I think I've found the problem though. I have a TCP wrapper that uses a value called "format" that can be 0, 1, or 2. If it is 0 (which in my program, it is), there is an unsigned short attached to the message and detached on the other end for size checking reasons. I think that since I'm receiving the message in two parts but only sending it once, it may be taking 2 bytes out both times and therefore I'm short 2 bytes.

Sadly, I didn't realize this was the problem until just now, and I don't get home for another 8 hours to test my theory. I'll report back when I find out.

[Edited by - ShinkaFudan on February 3, 2010 11:11:35 AM]

It appears you are using different packing instructions.

In c++ a structure without specifying a packing algorithms has packing header wich sometimes is 4 to 16 bites.


See this example:

struc myStructure
{
int i;
int a;
};



sizeof(myStructure) is not 8 bits, it could be 12 16 or even 24 depending the compiler!

Quote:

Original post by becoolnike It appears you are using different packing instructions. In c++ a structure without specifying a packing algorithms has packing header wich sometimes is 4 to 16 bites. See this example: struc myStructure { int i; int a; }; sizeof(myStructure) is not 8 bits, it could be 12 16 or even 24 depending the compiler!

I should have been more specific. This could not be the problem, because I am just putting pure data types into the stream. char, int, and float only. Also, I am using the same compiler and OS for both the server and client, so the sizes of the data are the same. Thank you for this though. This reminder about structure padding may prevent a problem later in my project.

Alright, I went home and tested my theory and it was correct. I wouldn't have thought of it without all of your help, so thank you very much.

For future notice:
- If a TCP packet gets an int put at the front of the buffer before being sent off, to be read when a piece of data is received, then make sure your system doesn't allow the reading to happen twice in a way that pulls the data out, as the second piece of the split packet won't have the int tied to it, and you'll end up trashing your data.

Would a simpler way to read your note be this?

"Each call to recv() or read() will remove data from the head of the stream, leaving less data for further reads."

Quote:

Original post by hplus0603 Would a simpler way to read your note be this? "Each call to recv() or read() will remove data from the head of the stream, leaving less data for further reads."

Yes, but the main point I was trying to make is that you can't assume a packet won't be split in two, so if you have a system that calls recv() on each part coming in, it will take data from the second half of a split message and screw up your data, especially if you are taking an int from the stream that you use to create a buffer based on the size.... you would grab junk data and possibly create gigs of space in a buffer.

Quote:

Original post by ShinkaFudan Yes, but the main point I was trying to make is that you can't assume a packet won't be split in two, so if you have a system that calls recv() on each part coming in, it will take data from the second half of a split message and screw up your data, especially if you are taking an int from the stream that you use to create a buffer based on the size....

This is what the word "stream" means. There are no packets in standard TCP stacks that caller could make use of.

Quote:

you would grab junk data and possibly create gigs of space in a buffer.

First rule of reliable and secure networking - never allocate resources in response to network requests.

If that is not viable, ensure that worst-case scenario has upper bound. If you do need to allocate a buffer, give it upper bound (each client can allocate 10 x 10,000byte buffers - if exhausted, start ignoring their input, disconnect them, stall them, enqueue up to maximum of 20 more messages without processing, ....). Many frameworks are dangerous in this respect, since they are designed without this in mind. So while they might not crash directly, they'll just degrade in unknown manner, until finally some higher level logic starts failing. Often this is incorrectly classified as scalable design, whereas it is more of a holistic approach - they just assume typical use case will fit entirely into available hardware.

This applies to much more than just a simple buffer, same principle needs to be applied accepting connections (number of active sockets/connections), the response/notification logic (can one client trigger all others to be notified, how to prevent or control n^2 case), the buffer sizes (string, container sizes, counts of any kind), and much more.

In networked environment, a corrupt 'size' member can cause a remote node to shutdown and deplete resources, and even cause a cascading failure. Trivial example is chat spam in busy chatroom. Each message is sent to n clients, so just a handful of malicious/broken clients can bring arbitrarily large system to its knees.

I suppose I have a lot to think about with my network architecture before I keep moving on with my project. Thank you for all of your help.

I highly recommend reading the Forum FAQ. Doing so would have saved you the pain of this particular bug, because it's already described in there :-)

Quote:

Original post by hplus0603 I highly recommend reading the Forum FAQ. Doing so would have saved you the pain of this particular bug, because it's already described in there :-)

Are you referring to Q14? It really doesn't mention this particular problem outright. I've been through the FAQ. It sort of mentions the root cause, but not the problem it creates. I'm sure if I'd sat and thought about each question long enough, I would have been able to foresee this problem, but I just read over it to make sure I wasn't asking a question that was already covered specifically. Sorry.