🔒 Locked

Data File Sizes

Started by MatrixCubed Dec 12, 2005 at 10:00 AM 8 replies 1k views

Original Post

Dec 12, 2005 at 10:00 AM

Hey all, I'm creating some data files for my game, and one of the problems I have encountered is structure byte-alignment. For example, the map-tile structure I use is 6 bytes in size (an unsigned integer and a signed short integer), however when I write this file to disk, e.g.


struct
{
    unsigned int type;
    short height;
} tile;

tile t;
fwrite(&t, sizeof(tile), 1, outfile);

... the size of a data file written to disk indicates a byte-alignment of 8, rather than 6. Is there an elegant method of handling this, given that I am designing my game engine cross-platform (specifically Win32, Linux, and eventually Mac OS X, all on 32 bit CPU platforms)? Currently I am writing out individual structure members (e.g. one fwrite() each for 'type' and 'height' in the example above), however if I follow this method throughout my game, anytime a struct's/class's structure is revised, it means rewriting a lot of code. I'm aware of #pragma pack(n) (where n is the byte boundary to use), however is this a trustworthy method of handling this problem? Thanks for any input :)

DrEvil

Dec 12, 2005 at 10:11 AM

#pragma pack(1) should pack it to 1 byte alignment, so 6 bytes for your struct.

#pragma pack(1)struct{    unsigned int type;    short height;} tile;// go back to default#pragma pack()

Aiursrage2k

Dec 12, 2005 at 10:16 AM

You could just have a define

#define TILESIZE 6
fwrite(&t, TILESIZE, 1, outfile);

MatrixCubed

Dec 12, 2005 at 10:19 AM

Quote:
Original post by DrEvil
#pragma pack(1) should pack it to 1 byte alignment, so 6 bytes for your struct.
#pragma pack(1)struct{    unsigned int type;    short height;} tile;// go back to default#pragma pack()

Unfortunately, the gcc/g++ compilers restrict structure packing to powers-of-two (defined here), but I'll give this method a look. Thanks :)

(Edit: Found a conflicting article here so it looks like I'll have to do a bit of testing...)

DrEvil

Dec 12, 2005 at 10:22 AM

Hmm, is that power of 2 restriction somewhat new? I've been using it with gcc 2.95 for some time now.

Kitt3n

Dec 12, 2005 at 10:28 AM

>You could just have a define
>#define TILESIZE 6
>fwrite(&t, TILESIZE, 1, outfile);

Be carefull with that.. I could image in a situation like
this it might give problems:

struct
{
char a
int b
char c
int d
} ;

Best might be to try to manually put everything on 4 byte
borders (by cleverly arranging your data-members), because
if you use pack to put everything on byte boundaries at
runtime it might slower...

RobTheBloke

Dec 13, 2005 at 7:36 AM

1 is a power of 2 (2^0). the #pragma pack(1) will work correctly with gcc. That will work correcty on mac/linux & win32 fine. The only issue with mac is that the byte order is different so you will have to do the LSB/MSB conversion for that platform. I normally use a cunning union to do that....

template<typename T>struct _Convert2 {  union {    T value;    unsigned char bytes[2];  };  _Convert2(const T& v) { value = v; }  void flip() {    unsigned char temp=bytes[0];    bytes[0] = bytes[1];    bytes[1] = temp;  }};template<typename T>struct _Convert4 {  union {    T value;    unsigned char bytes[4];  };  _Convert4(const T& v) { value = v; }  void flip() {    unsigned char temp=bytes[0];    bytes[0] = bytes[3];    bytes[3] = temp;    temp=bytes[1];    bytes[1] = bytes[2];    bytes[2] = temp;  }};#ifdef MACOStemplate<typename T>void Convert2(T& t) {  _Convert2<T> conv(t);  conv.flip();  t=conv.value;}template<typename T>void Convert4(T& t) {  _Convert4<T> conv(t);  conv.flip();  t=conv.value;}#else // do nothing for linux or win32template<typename T>void Convert2(T& t){}template<typename T>void Convert4(T& t){}#endif// so to read/write the data do :struct{    unsigned int type;    short height;} tile;void WriteTile(tile& t){  // make sure fields are flipped before writing  Convert2(t.height);  Convert4(t.type);  fwrite(&t, sizeof(tile), 1, outfile);  // just incase we still need the data afterwards  Convert2(t.height);  Convert4(t.type);}void ReadTile(tile& t){  fread(&t, sizeof(tile), 1, outfile);  // ensure flipped byte order for Mac  Convert2(t.height);  Convert4(t.type);}

[Edited by - RobTheBloke on December 13, 2005 7:36:09 AM]

cruppstahl

Dec 13, 2005 at 9:37 AM

There's a format which was created just for transferring data without handling endian problems, packing etc. It's called ASN.1. It might be an overkill to write an ASN.1 parser for simple structures, but maybe you can steal some ideas, since ASN.1 is quite fast and works great.

RobTheBloke

Dec 13, 2005 at 9:40 AM

you could of course just use ascii since that avoids the problem alltogether. The obvious downside is parsing speed. I normally just assume that mac users will have to put up with a bit of overhead.... (since Win32 has the most users, i like to keep the majority happy)

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.