Skip to main content
gamedev.net
🔒 Locked

Data storage C

Started by Tquila Nov 14, 2011 at 10:36 PM 11 replies 5.6k views
Original Post
Tquila
Tquila
First off, I don't know if this was better placed in "General Programming" then please move it.

Anywho - I got a pretty good understanding of general programming now. I'm closing in on the exam in imperative programming ( in C ), and I've got a quite good understanding on programming in general. So I thought I'd try and make a text based RPG, and I don't really have an issue with most of the things I'll be making (functions etc), but one thing we havn't quite touched in lectures yet is data storage (more so than just touching the file I/O). My plan was to create items in structs (possibly a general item struct and then some more specified item structs for various types of items, e.g. a struct for weapons which holds min/max dmg, enhancements (which specifics would then be held in another struct) etc).

I've thought about doing it with arrays too, but structs has hit me like the best solution. However, saving these data values are obviously key, and my initial idea was to store it in a .txt or equal file, where I could then acces it through the file I/O. I may be totally off course here, as there may be better practices (please do enlighten me, only going on what I've learned so far). I realize many people write this in C++, but my main priority here is to stay in C, and I thought making a larger project would enhance my knowledge of the general software engineering principles even more (not to mention practice makes perfect and repetition gives "on the top of the head know how").

I've been looking around, but as said most of these are in C++.

TL;DR: General programming patterns for saving/retrieving data in C, and general structure of files etc of a project of this type.

If any elaboration is needed please ask.
dpadam450
dpadam450
http://www.java2s.co...ingfromfile.htm
http://www.cplusplus...library/cstdio/

Just do something like this, code is not 100% but you can search for it online.


struct weapon
{
int damage;
int num_bullets;
int num_clips;

void WriteWeapon();
void LoadWeapon();
};

void WriteWeapon()
{
//open a file for writing
fprintf(out_file, "%i", damage);
//.....print other properties
}

void LoadWeapon()
{
//open file to read
[color="#7f0055"]char data[color="#000000"][[color="#990000"]128[color="#000000"]][color="#000000"];
fgets(in_file, data);
int damage = atoi(data);//convert characters to an int
}
NBA2K, Madden, Maneater, Killing Floor, Sims 
zacaj
zacaj
You probably want to use either plain text or use fwrite to save whole structs (or arrays of structs) to files.
My Site

@zacaj_
WahidB
WahidB
Use binary to write your files http://www.cprogramm...al/cfileio.html

struct Item
{
int id;
int level;
char name[64];
};

struct Item item;

//Write
FILE* pFile;
fopen_s(pFile, "filename", "wb");
fwrite((void*)&item, sizeof(item));
fclose(pFile);

//Open
fopen_s(pFile, "filename", "rb");
fread_s((void*)&item, sizeof(item), 1, sizeof(item), pFile);
fclose(pFile);

something like this.
BeerNutts
BeerNutts

Use binary to write your files http://www.cprogramm...al/cfileio.html

struct Item
{
int id;
int level;
char name[64];
};

struct Item item;

//Write
FILE* pFile;
fopen_s(pFile, "filename", "wb");
fwrite((void*)&item, sizeof(item));
fclose(pFile);

//Open
fopen_s(pFile, "filename", "rb");
fread_s((void*)&item, sizeof(item), 1, sizeof(item), pFile);
fclose(pFile);

something like this.


Yes, it's going to be much easier to read and write your structures in binary than trying to convert to and from text. I'm not sure the "-S" is need for most C compilers, but in general you'll want to use:
fopen
fread
fwrite
fclose

You mention deciding between array's and structures, but you can use both;

typedef struct
{
int Damage;
int Cost;
int SwingSpeed;
char Name[64];
} tWeapon;

#define MAX_PLAYER_WEAPONS 8

// An array of weapons the player can own
tWeapon gPlayerWeapons[MAX_PLAYER_WEAPONS];
int gNumPlayerWeapons = 0;


My Gamedev Journal: 2D Game Making, the Easy Way

---(Old Blog, still has good info): 2dGameMaking
-----
"No one ever posts on that message board; it's too crowded." - Yoga Berra (sorta)
dpadam450
dpadam450
The only reason I advised possibly starting with text is:

it is readable to find flaws for a beginner. Can also be manipulated by hand if he needs to change anything. If he has pointers to upgrades or any pointers in the structs he won't be able to write out the structure as posted. So to the OP: if you have any pointers in those structs, you will have to grab the objects out of the pointers and reference their actual data members. If you think you might add pointers, you can still do binary, but you will have to do it each variable at a time.
NBA2K, Madden, Maneater, Killing Floor, Sims 
Cornstalks
Cornstalks
What you're looking for is a way to serialize data. The exact structure and format of a particular file is entirely up to you and can be changed fairly easily, but one challenge is serializing the data. As you've seen, there's lots of C++ libraries for this, but less C libraries. This guy had a question very similar to yours, and two of the top recommended libraries from that thread are Eet and Tpl. Note that they say Tpl can't store nested data structures, though Eet can. It all depends on how your structures are organized.

You could also write your own serialization functions for each data structure if you want, though it's up to you. One thing to be careful about that a lot of the solutions posted here in this thread have a problem with is endianness and alignment issues. Data structures can be stored differently in memory across different computers. Taking a direct copy of the memory from one computer, saving it to disk, and then trying to load it on another computer can create issues. That's why a good serialization library is important (whether you write your own or you use another existing library), as it avoids these issues.
Mallach
Mallach
It definitely would be a good exercise to hone your current dev and programming knowledge.

Just throwing this out there. I don't know what your total background is. You might want to check out SQLite if you have some background with databases. It will save you a lot of trouble writing your own file formats. I use it extensively in my roguelike/RPG to store and organize game data.
Vortez
Vortez
I think it would be better if you save them in a binary file, cause this way there's no parsing to do, you just read or write the structure data and you're done. Use fopen(fname, "rb";); or fopen(fname, "wb+";) to open/create a file, fread/fwrite to read and write and you should be fine. Of course don't forget fclose() when you're done.

Ex:

struct float3 {
float x, y, z;
};

float3 data;

// Write it to a new file
FILE *f = fopen(fname, "wb+");
fwrite(&data, 1, sizeof(float3), f);
fclose(f);

// Read it back from the file
FILE *f = fopen(fname, "rb");
fread(&data, 1, sizeof(float3), f);
fclose(f);
Cornstalks
Cornstalks
Vortez, what if he wants to store multiple objects of potentially different types? What you posted there doesn't allow for that, plus it requires him to never recompile to program if he is ever going to read from the same file. Parsing and serialization is necessary if writing out to a binary format that's intended to last for more than one revision.

I think Mallach's idea is a brilliant one. Probably the simplest as well.
zfvesoljc
zfvesoljc
why not serialize with xml? It's human readable, editable and there are some nice free xml libraries out there than can be used. I've used tiny xml before, haven't found any serious issues with it.


just a hint, if you have bitwise flags in an attribute, don't save the attribute as a value, save each flag on it's own. It can save you a lot of trouble


enum EAttr {
aActive = 1,
aVisible = 2,
aMagic = 4,
};

int attr;
// DONT
xml.Write( "attr", attr );
// DO
xml.Write( "active", attr & aActive );
xml.Write( "visible", attr & aVisible );
xml.Write( "magic", attr & aMagic );
Hodgman
Hodgman

Vortez, what if he wants to store multiple objects of potentially different types? What you posted there doesn't allow for that, plus it requires him to never recompile to program if he is ever going to read from the same file. Parsing and serialization is necessary if writing out to a binary format that's intended to last for more than one revision.
Storing different data types, or even complex collections of types is indeed possible in such a simple serialisation system.
The proprietary engine that I use at work is makes use of this serialization strategy extensively. The biggest advantage for us is that any data-structure (e.g. arrays, polymorphic types, hash-maps, linked-lists, trees etc...) can be stored this way and used immediately after loading (with no parsing step) -- imagine if loading/saving an entire level of data was done by a single memcpy/fread/fwrite call!

To get around the fact that the binary files can't be ported between revisions, the editing tool-chain all operates on other formats, such as XML, and compiles content files into binary for use by the game. If a file-format changes during development, the tool-chain can automatically re-build the affected data files.

Actually, one of the great things about C is that everything is just made up of bytes, so you can memcpy anything (except pointers, because they point to specific addresses) from one location to another, and it still works. "Another location" even includes compressed files or the network.

Because pointers can't be memcpy'ed, this means you actually do need a parsing stage, which updates pointers with appropriate values. e.g.struct Player { const char* name; };
struct World
{
int numPlayers;
Players* players;
};

char* dataLoadedFromFile = ...;
World* world = (World*)dataLoadedFromFile;
int offset = (int)world->players;//in the file, instead of using pointers, use offsets from the beginning of the file
world->players = (Player*)(dataLoadedFromFile + offset);
for( int i=0; i!=world->numPlayers; ++i )
{
offset = (int)world->players.name;
world->players.name = dataLoadedFromFile + offset;
}However, you can remove the need for this parsing step if you instead calculate the appropriate pointer values on-use, e.g.struct Player { u32 offsetName; };
struct World
{
int numPlayers;
u32 offsetPlayers;
};
const char* GetPlayerName( World* world, int index )//the data pointed to by world may have just been read straight from disk
{//instead of pointers, relative offsets from the 'pointer' variable to the data allocation are used
Player* players = ((char*)&world->offsetPlayers) + (int)world->offsetPlayers;
return ((char*)&players.offsetName) + (int)players.offsetName;

However, despite how great binary data-files are wink.gif, I'd still actually recommend that a beginner should use a text format such as JSON/XML/CSV/etc... as it's pretty important to become familiar with generic serialisation.
Krohm
Krohm

why not serialize with xml?
Thank you very much. I think your question is the most value added thing here (+1 for you).
Let me elaborate on your proposal, for the sake of an hopefully informative discussion.

As some people on this forum will say - or maybe not - data storage/serialization comes in several flavors. The first problem is evaluating the amount of data and its complexity. We then have a broad spectrum of choices. If there's a lot of data to be deployed to final product we have little choice: binary will likely be the best - there could still be some cases in which a text representation might work better, and I think this is one of those cases.
Even having hundreds of item templates, hundreds of monster templates and thousands of text locations, the data footprint involved seems to be very limited. At this point, I'd consider text representation.
This comes in essentially two flavors.
  1. JSON is probably the simplest. Integrating the library took 1 day for me (although it took me 1 week to choose what lib to use). JSON is indeed a sweet spot for data storage. Its requirement of having quoted strings helps in giving the data files a "formal" look and feel. The user will be able to edit them but will also immediately recognize them as "restricted" by some rules. That is, in my opinion very good. JSON has demonstrated to be more than enough for most applications although it lacks some features such as object inter-referencing.
  2. YAML is somewhat compatible with JSON (some JSON files are YAML files and vice versa, JSON is supposed to be a subset of YAML). Not only YAML leverages some restrictions with JSON, it also provides more functionalities and more flexible syntax. It appears the homepage is YAML compliant... I have mixed feelings about YAML. It appears that the increased flexibility in syntax might eventually encourage users in messing up more (with all the pros and cons). If the data is simple, you probably don't need YAML. If the data is complex, you probably want to bite the bullet and go for XML.
  3. XML has demonstrated to be largely able to cope with all the data you can throw at it. It is not a coincidence whole industry standards such as Collada are based on it. Nonetheless, it appears to me XML is a "final solution" which is probably too broad for this task. I also have some doubts XML is "human readable", the S/N ratio becomes just bad as the files grow. Tag-based stuff only works as a certain point.

So, I'll follow the suggestion I have read by other people in this forum: if it's data, it's either JSON or XML. If it's code, it's Lua, Python or some other language.
Parsing strings are just as boring as producing them. All the above choices do that for you.

EDIT: This message lost the race against Hodgman's.
Previously "Krohm"

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.