🔒 Locked

Efficient Packet Structures - Help?

Started by kurifu Dec 27, 2001 at 5:08 PM 29 replies 3.8k views

Original Post

Dec 27, 2001 at 5:08 PM

I have started to think about how I am going to format the incoming data packets (both UDP and TCP as the server will rely upon both protocols in different circumstances), and I seem to have come down to a few options.. however I am not presently satisfied with their efficiency and would like to ask for advice on more efficient packet stucturing, transmission, and receival. String Parsing Technique: I could just append all of the data, using wsprintf, into a text string and send that data. When the server receives this data it will parse the string, figure out what information is in the string, and make the appropriate changes. Only thing I do not like about this is that there will be an immense amount of work to parse the string and collect the data from it. Structure Tecnique: Well, as an attempt to not have to parse the strings from above, my other solution would be to create a structures for each type of outgoing packet, fill the structures, and read them back on the server end. The data is already sorted in the structure, however the problem that poses is... what structure am I supposed to be receiving? What happens if the packet is too large for the client? Another attempt at this would be to create a generic structure type with two fields, one a word, and the rest an array of char so that any data can be easily passed into it, and when the packet type is identified from the word, the data can be recasted into its proper structure type. Only problem I see here is that the full array of char will be transmitted, and may only contain 2 bytes of relevant data... this is a waste of bandwidth. I am not even entirely sure how much MSVC++ will like recasting the structures at that... Now, if I were to change that array of char to a pointer of char in the packet, though the pointer can hold a dynamic quantity of information, the problem comes that when you try to send the structure, the pointer address will be passed and not the information that it is pointing to (and will lead to a nice mess of random or corrupt data, and likely even an access violation). Anyone know of any good solutions to this problem or have any ideas? All help will be greatly appreciated. Sincerely, Clifford M. Roche Gamedev''s AI Auto-Reply bot.

KalvinB

Dec 27, 2001 at 5:17 PM

Use byte code.

sending
unsigned char message[maxmessagelength]

message[0]=40 ; //message type is 40 (player position)
message[1]=30 ; //player number 30
message[2]=100; //xpos = 100
message[3]=20; //ypos = 20

send(message, 4); //send four bytes of the message array

reciveing
switch(message[0])
{
case 40:
player[message[1]].xpos=message[2];
player[message[1]].ypos=message[3];
break;
}

I think you get the idea. The hard part is keeping track of what each byte of each message type represents.

Ben

kurifu

Dec 27, 2001 at 5:24 PM

That works along the lines of using the WORD header to describe the packet, and the array of char to send the data, only without recasting that data...

But the problem is that MAXMSGLEN could be (for example) 256bytes.

So when you send ( message, 4 ); you are truncating the outgoing structure.

When the server receives the data and you call

recv( SOCKET, &buffer, MAXMSGLEN ); you are trying to receive the entire buffer size, but will only be reciving part of it, will this not corrupt the data?

Gamedev''s AI Auto-Reply bot.

kurifu

Dec 27, 2001 at 5:25 PM

I get it now...

And if I am concerned about data corruption, I can just memset ''\0'' the structure before receiving...

It bothers me that the solution was so simple, yet I neglected to find it...

Thanks in any case

Gamedev''s AI Auto-Reply bot.

kurifu

Dec 27, 2001 at 11:41 PM

No, it seems that I can not recast these two structures:


typedef struct FOO1 {
     WORD PacketID;
     unsigned char Data[100];
} UNIVERSALTCPPACKET;


typedef struct FOO2 {
     WORD PacketID;
     char Data1[50];
     char Data2[50];
}


How do games and game servers normally communicate with one another and avoid this problem?

I think I am going to go read a little and see what other information I can find.

Gamedev''s AI Auto-Reply bot.

KalvinB

Dec 28, 2001 at 12:53 AM

Tombstone uses the method I proposed and it works great. Winsock should report the size of the incomming packet.

Have you tried testing it to see if data is actually corrupted? For DPlay I don''t have to do anything special. It''s just basically doing a memcpy from the packet to the struct and if less comes in than space is avaiable then it doesn''t care.

Ben

fingh

Dec 28, 2001 at 5:20 PM

Kurifu,

Just use a byte stream. Pack your structure members into a byte array, in a known order, so that the receiving end can unpack in the same order. Attach a header so you know the total size and the type of message. The actual code that does the recieving shouldn''t need to know about what it is receiving. This is how some commercial MMORPGs do it.

Dire.Wolf

Dec 28, 2001 at 8:00 PM

Fingh,

I have to disagree slightly. The receiving code needs to know a little bit about what is being transmitted if you are implementing guaranteed messaging via UDP. You need to check for data corruption/incomplete messages and manage ACKs for packets marked as guaranteed.

Obviously this could be abstracted to receiving the actual packet and the decoding of the received data.

Dire Wolf
www.digitalfiends.com

kurifu

Dec 28, 2001 at 10:59 PM

fing, essentially that is what I was doing, only I am casting the bytestream to a structure to more easily access the data, and have a universal header for all packets so that I may properly identify data such as wether to respond with an AK, and what kind of data is coming in (to know which packet to caste it to).

Gamedev''s AI Auto-Reply bot.

fingh

Dec 29, 2001 at 12:55 PM

You don''t really need to know what the data is, you just need the data itself to recalculate the checksum. This would obviously exclude the packet header (and therefore the sequence field and the crc) which he''s already determined to be of known size and composition. The difference, as you noted Dire, is that the header would be unpacked in the receiving code, but not necessarily the data itself (which is what I meant). Sorry if I caused some confusion, but I thought (based on previous posts) that the use of a packet header was implied.

Goblin

Dec 30, 2001 at 2:58 PM

For packets that are going to be of variable length, couldn''t you put the length as the first value in the packet, peek recv that into a small buffer, and then create a buffer of the appropriate size to actually recv it to?

-----------------
The Goblin (madgob@aol.com)
-----------------
"Before critisizing somebody, walk a mile in their shoes. That way, when you do critisize them, not only will you be a mile away, but you''ll also have their shoes!"

Do00d

Dec 30, 2001 at 7:00 PM

nope thats a bad idea
when u recieve a packet and u specify a buffer and its lenght winsock takes that value either as the size of the buffer or how much it should recieve
the thing is to make a big enough arraysay char buf[2048] unless ull send more
and that way ull never have the problem of getting something too big

but on another note or question actually
say im making quake 4 or UT2
how did them ppl handle sending things like rockets
did they first create a predertermined array and then fill it with the data and then send all of it or part of it?
or make a linked list and then send the data there?
if it was completly dynamic and u had like 16 - 32 players to worry about wouldnt it be kind of slow having to create a new buffer on the free store just to send a packet?
this is a more overall desing question

ppl seem to be always talking bout sending this snapshot every so and so milliseconds but since different data will be sent to each player the packets would vary in size so that would mean like 32 allocations and deallocations every 20 - 50 milliseconds
and tahts just for sending
am i missing something here?

kurifu

Dec 30, 2001 at 7:24 PM

You are going to send a simple array of data that tells the server that you would like to launch a rocket... very simple, t echnically this only requires one byte of data.

Since the server should be keeping track of your mouvement in mych the same pattern, if you can actually launch a rocket (ie the server tests to see that you have a rocket launcher, and enough munition ), then it returns data to all the computers representing the speed, direction, and origin of the rocketr... the clients than draw it.

The server will also keep track of the collision detection to tell you when someone has been hit, not the client... as this avoids cheating.

Gamedev''s AI Auto-Reply bot.

kurifu

Dec 30, 2001 at 7:32 PM

Yes, the packets will be of a varying size, but there will be a structure andf MAX size to it, and hence casting data structures is an easy, effective, and perfectly acceptable manner to complete.

In response to whomever posted to send the size in the packet, that can be done, but should I make reference to the SSHv1 packet hacks that have been going around? I feel that it would impair the security of the packets to do so, especially when there is another way around it.

CRC checks will be shortly implemented into the packets.. I just have to write a few functions, I will also be providing a basic dual key encryption on the packets to in the near future to increase their security.

Gamedev''s AI Auto-Reply bot.

JonStelly

Dec 31, 2001 at 1:32 AM

People like to get way too complicated with this. Here''s my basic format:

  
//MESSAGE TYPES

#define MSG_SAY    0x01

//MESSAGE LENGTH CONSTANTS

#define MAX_SAY    256

typedef struct NETMSG{
    BYTE bFlags;//PACKET FLAGS (GUARANTEED,ACK)

    USHORT nId;//PACKET ID

    USHORT nType;//PACKET TYPE ID

    USHORT nLength;//PACKET LENGTH

    NETMSG(USHORT type){
        nType = type;        
    }
}*PNETMSG;

typedef struct NETMSG_SAY : NETMSG{
    USHORT nCharacterId;//WHO SAID IT

    TCHAR szText[MAX_SAY+1];//WHAT THEY SAID

    NETMSG_SAY(USHORT character, LPTSTR text) : NETMSG(MSG_SAY){
        _tcsncpy(szText, text, MAX_SAY);
        nLength = sizeof(NETMSG) + sizeof(USHORT) + ((_tcslen(szText) + 1) * sizeof(TCHAR));
    }
}*PNETMSG_SAY

Then when you send the packet, use the nLength member to determine how many bytes of your packet to actually send. On the receiving side, take your received packet, cast it to a (PNETMSG) and use this structure to test the nType member. When you figure out what type of packet it is, cast it to the correct message type and handle accordingly.

kurifu

Dec 31, 2001 at 5:56 PM

Only one problem I see with that is that because you have the length stored at the very end, you will have to receive all of your data string, even if only 1 byte of it is used.

This is what I am trying to avoid.

Gamedev''s AI Auto-Reply bot.

kurifu

Dec 31, 2001 at 6:12 PM

Could someone also explain to me how you get those nice looking code blocks like above?

Gamedev''s AI Auto-Reply bot.

Dire.Wolf

Dec 31, 2001 at 11:16 PM

Type:

[_code_] and [_/code_]

or

[_source_] and [_/source_]

but take out the underscores (_)

BTW Normally you receive all your data before using it, especially considering that UDP is message based.

Regards,

Dire Wolf
www.digitalfiends.com

Edited by - Dire.Wolf on December 31, 2001 12:17:33 AM

JonStelly

Jan 1, 2002 at 1:10 AM

Actually, the nLength member appears at the front of the packet. When you derive a class / structure , the members of the children are added to the end of the memory area of the parent.

With the packet structure I have above, if you place any variable length fields at the end of the packet structure, you can reduce the number of bytes transmitted. You still allocate a structure which will allocate all of the bytes on the sending side, but you only have to send enough bytes to send the valid data.

kurifu

Jan 1, 2002 at 4:49 PM

Ah... I see it now

Essentially the same idea that I was going for. I did finally complete my code and tested it. It works without fail... though I have to now start taking into consideration problems such as splitting the packets, and so forth.

None the less though what I have for now does work over my LAN, and in the near future I am going to write the packet disassembly and reassembly code. Is there any chance that the data itself will be split up somewhere along route?

Gamedev''s AI Auto-Reply bot.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.