🔒 Locked

Save Files/Load Time question

Started by veigue Jul 2, 2009 at 4:28 PM 21 replies 3.7k views

Original Post

Jul 2, 2009 at 4:28 PM

I'm working with a group on a Final Fantasy style rpg right now and we have a few concerns regarding our save files. Due to the number of values we'll need to record in our save files, some of our members are worried that the save files will be too large/will take too long to load. One major concern is our Encyclopedia, which fills in entries on monsters based on how many of each the player has killed and requires us to keep track of kill counts of every enemy. Aside from this, it's pretty much your typical rpg stuff -- character data, inventory, progression, etc. Since none of us have any experience creating such detailed save files, we don't really know what to expect, so I was wondering if anyone with experience could tell us whether load times will even be a problem and, if so, could maybe suggest some creative ideas for storing our data to help with these issues. We are currently planning on using text files for our save data and they will probably be encrypted later on. Any input would be much appreciated and sorry if I made a wall of text...

Zipster

Jul 2, 2009 at 4:52 PM

Save games aren't really something you want to save in text format, since it becomes very easy for the player to edit them and cheat. If you encrypt the files this won't be a problem, but text is also generally slower to load because it takes more memory to express information than binary, and you have to parse the text for meaning, which takes time as well. The rule of thumb is if you don't need the data to be human-readable, then make it binary.

Unless your save games are dozens of megabytes, load times shouldn't be an issue. The real bottleneck come from initializing the game from this data once it's loaded, not the loading itself. That is usually dependent on your games general load times and has nothing to do with saved games in particular. You'll often notice in many games that loading saved games is a lot faster once you're already in a game than it is at the main menu. And some games just re-initialize everything. It depends on the game.

There's really nothing you can do until you implement save/load and find it's too slow, then you can optimize. So for now I wouldn't worry about it :)

Mybowlcut

Jul 2, 2009 at 4:58 PM

I'd firstly suggest XML instead of a plain text file. It is much more consistent and easier to implement than trying to hack up your own text parser. Try tinyxml or google around for C++ XML parsers.

Secondly, you need to determine which information can be cached and shared among objects. If you can assign a short ID to this information, then it means only an integer has to be stored in both the XML and the program code. If you're creating a Final Fantasy clone (I'm assuming tile-based 2D), then you can have a tile cache which lists tiles and assigns them IDs; have a look at mine as an example:

 <?xml version="1.0" ?><tile_cache>    <tile id="1" name="thick_grass" file_name="Tiles\thick_grass.png" animated="0" is_colour_key="0" />    <tile id="2" name="thick_wet_grass" file_name="Tiles\thick_wet_grass.png" animated="0" is_colour_key="0" />     <tile id="3" name="desert_+_grass_+_water_l" file_name="Tiles\desert_+_grass_+_water_l.png" animated="0"is_colour_key="0" />    <tile id="4" name="desert_+_grass_tl" file_name="Tiles\desert_+_grass_tl.png" animated="0"is_colour_key="0" />    <tile id="5" name="desert_+_grass_t" file_name="Tiles\desert_+_grass_t.png" animated="0" is_colour_key="0" />    <tile id="6" name="desert_+_grass_tr" file_name="Tiles\desert_+_grass_tr.png" animated="0is_colour_key="0" />    <tile id="7" name="desert_+_grass_bl" file_name="Tiles\desert_+_grass_bl.png" animated="0"is_colour_key="0" />    <tile id="8" name="desert_+_grass_b" file_name="Tiles\desert_+_grass_b.png" animated="0" is_colour_key="0" />    <tile id="9" name="desert_+_grass_br" file_name="Tiles\desert_+_grass_br.png" animated="0"is_colour_key="0" />    <tile id="10" name="shallow_water" file_name="Tiles\shallow_water.png" animated="0" is_colour_key="0" />    <tile id="11" name="deep_water" file_name="Tiles\deep_water.png" animated="0" is_colour_key="0" />    <tile id="12" name="shallow_water_+_grass_b"file_name="Tiles\shallow_water_+_grass_b.png" animated="0" is_colour_key="0" />    <tile id="13" name="desert_+_grass_l"file_name="Tiles\desert_+_grass_l.png" animated="0" is_colour_key="0" />    <tile id="14" name="desert" file_name="Tiles\desert.png" animated="0" is_colour_key="0" />    <tile id="15" name="desert_+_grass_r" file_name="Tiles\desert_+_grass_r.png" animated="0" is_colour_key="0" />    <tile id="16" name="a_shallow_water_n"file_name="Tiles\a_shallow_water_t.png" animated="1" qty_frames="4" wrap="1" progression="FORWARD" frame_delay="0.7" is_colour_key="0" />    <tile id="17" name="a_shallow_water_s"file_name="Tiles\a_shallow_water_b.png" animated="1" qty_frames="4" wrap="1" progression="FORWARD" frame_delay="0.7" is_colour_key="0" />    <tile id="18" name="a_shallow_water_e"file_name="Tiles\a_shallow_water_r.png" animated="1" qty_frames="4" wrap="1" progression="FORWARD" frame_delay="0.7" is_colour_key="0" />    <tile id="19" name="a_shallow_water_w"file_name="Tiles\a_shallow_water_l.png" animated="1" qty_frames="4" wrap="1" progression="FORWARD" frame_delay="0.7" is_colour_key="0" />    <tile id="20" name="a_fire" file_name="Tiles\a_fire.png" animated="1" qty_frames="5" wrap="0" progression="FORWARD" frame_delay="0.35"is_colour_key="1" />    <tile id="21" name="shallow_water_+_grass_t"file_name="Tiles\shallow_water_+_grass_t.png" animated="0" is_colour_key="0" />    <tile id="22" name="shallow_water_+_grass_l"file_name="Tiles\shallow_water_+_grass_l.png" animated="0" is_colour_key="0" />    <tile id="23" name="shallow_water_+_grass_r"file_name="Tiles\shallow_water_+_grass_r.png" animated="0" is_colour_key="0" />    <tile id="24" name="shallow_water_+_grass_br"file_name="Tiles\shallow_water_+_grass_br.png" animated="0" is_colour_key="0" />    <tile id="25" name="shallow_water_+_grass_bl"file_name="Tiles\shallow_water_+_grass_bl.png" animated="0" is_colour_key="0" />    <tile id="26" name="shallow_water_+_grass_tl"file_name="Tiles\shallow_water_+_grass_tl.png" animated="0" is_colour_key="0" />    <tile id="27" name="shallow_water_+_grass_tr"file_name="Tiles\shallow_water_+_grass_tr.png" animated="0" is_colour_key="0" />    <tile id="28" name="grass_+_shallow_water_br"file_name="Tiles\grass_+_shallow_water_br.png" animated="0" is_colour_key="0" />    <tile id="29" name="grass_+_shallow_water_bl"file_name="Tiles\grass_+_shallow_water_bl.png" animated="0" is_colour_key="0" />    <tile id="30" name="grass_+_shallow_water_tl"file_name="Tiles\grass_+_shallow_water_tl.png" animated="0" is_colour_key="0" />    <tile id="31" name="grass_+_shallow_water_tr"file_name="Tiles\grass_+_shallow_water_tr.png" animated="0" is_colour_key="0" />    <tile id="32" name="cave_entrance" file_name="Tiles\cave_entrance.png"animated="0" is_colour_key="1" />    <tile id="33" name="cave_exit" file_name="Tiles\cave_exit.png" animated="0" is_colour_key="1" /></tile_cache>

Once you've got that figured out, you can make a Tile_Cache class or similar:

class Tile_Cache: public boost::noncopyable, public ticpp::File_Readable{public:    typedef boost::shared_ptr<Tile> tile_ptr;    Tile_Cache(const std::string& file_name);    ~Tile_Cache();	// Returns the tile at ID.    tile_ptr Tile_At(unsigned int ID);        private: 	typedef boost::unordered_map<unsigned int, tile_ptr > cache_map;	typedef cache_map::iterator cache_it;    // Loads all the tiles at file_name into the cache.    virtual void Read(const std::string& file_name);	cache_map cache;};

Keeping data as small as possible in the XML file is what I tried to do (excuse the large code dump):

<?xml version="1.0" ?><level name="default_level_100x100" width="100" height="100">    <background_tiles tile_ids="18 18 18 ... etc ... " />    <object_tiles>        <object_tile name="test" file_name="Tiles\\a_fire.png" pass="2"animated="1" qty_frames="5" wrap="0" progression="BACKWARD" frame_delay="0.4" is_colour_key="1" pos_x="5" pos_y="7">            <auto_sound file_name="Audio\\fire.wav" volume="80" loops="-1" />        </object_tile>    </object_tiles>    <foreground_tiles tile_ids="0 0 0 ... etc ..." />    <passabilities passabilities="1 1 1 ... etc ..." />    <portals>        <portal to_level="Levels\\island_cave.xml"from_x="22" from_y="13" to_x="8" to_y="18" />    </portals>    <piles>        <pile pos_x="19" pos_y="12">            <contents>                <item_pair qty="3">                    <item id="0" />                </item_pair>            </contents>        </pile>        <pile pos_x="18" pos_y="14">            <contents>                <item_pair qty="4">                    <item id="1" />                </item_pair>            </contents>        </pile>    </piles>    <music>        <level_theme file_name="Audio\\test.mid" volume="10"loops="-1" fade_in="2.0" force_play="0" />    </music></level>

For some things you have to store the full representation, but for other things (like tile and even sound paths/image paths), you can save some space by referring to them with IDs.

A file with 100000 tiles loads quite quickly in the release build, so you shouldn't worry about it until you get to that stage.

Someone else might say that a database would be preferable, but I don't have any experience with integrating those with C++ so I'll leave it there...

veigue

Jul 2, 2009 at 5:45 PM

Thanks for the responses! They were very helpful :)

Developer_X

Jul 2, 2009 at 10:04 PM

I personally use binary files.

I usually write a save game editor tool that loads in save games for verification of properly saved data, and modification (i.e. cheating for faster testing) too.

For an example..

// main.cpp// Project: Binary Save Game File I/O Example// Author: Richard Marks <ccpsceo@gmail.com>#include <cstdio>#include <cstdlib>#include <cstring>class NPC{public:	NPC(const char* name, int x, int y)	{		name_ = strdup(name);		worldx_ = x;		worldy_ = y;	}		~NPC()	{		if (name_)		{			free((void*)name_);			name_ = 0;		}	}		void stats()	{		fprintf(stderr, "The NPC \"%s\" is at %d, %d.\n", name_, worldx_, worldy_);	}	const char* get_name() const {return name_;}	int get_world_x() const {return worldx_;}	int get_world_y() const {return worldy_;}		void set_name(const char* name)	{		if (name_)		{			free((void*)name_);			name_ = 0;		}		name_ = strdup(name);	}	void set_world_x(int x){worldx_ = x;}	void set_world_y(int y){worldy_ = y;}private:	const char* name_;	int worldx_;	int worldy_;};int main(int argc, char* argv[]){	// create the guy	NPC guy1("Guy #1", 50, 50);		// show his stats	guy1.stats();		// open the savefile for writing	FILE* fp = fopen("gamesave.dat", "wb");	if (!fp)	{		fprintf(stderr, "open save file (write) failed!\n");		exit(1);	}		// save the guy data	int x = guy1.get_world_x();	int y = guy1.get_world_y();	fwrite(guy1.get_name(), sizeof(const char*), 1, fp);	fwrite(&x, sizeof(int), 1, fp);	fwrite(&y, sizeof(int), 1, fp);		// close the file	fclose(fp);		// open the savefile for reading	fp = fopen("gamesave.dat", "rb");	if (!fp)	{		fprintf(stderr, "open save file (read) failed!\n");		exit(1);	}		// load the guy data	char savedname[1024];		int x2, y2;	fread(savedname, sizeof(const char*), 1, fp);	fread(&x2, sizeof(int), 1, fp);	fread(&y2, sizeof(int), 1, fp);		// close the file	fclose(fp);		// set the guy's member variables to the loaded data	guy1.set_name(savedname);	guy1.set_world_x(x2);	guy1.set_world_y(y2);		// show the stats again	guy1.stats();		return 0;}

Just my $0.02
Have fun! :)

prh99

Jul 2, 2009 at 10:34 PM

Quote:

One major concern is our Encyclopedia, which fills in entries on monsters based on how many of each the player has killed and requires us to keep track of kill counts of every enemy.

If your encyclopedia is stored in a set order you can use an array of characters or integers a use individual bits to represent whether each monster has been encountered. As for kill counts, just save out a series of integers in the same order as the monsters are listed in your encyclopedia.

I personally prefer binary files.

Nypyren

Jul 2, 2009 at 11:18 PM

XML will just make your loading times even slower, and your code more complex than it needs to be.

I recommend using binary. You don't have to parse anything, you don't have to worry about escaping quotes or whatever else, your code stays simple, and you use less drive space.

Developer_X

Jul 3, 2009 at 12:45 AM

Quote:
Original post by prh99
Quote:

One major concern is our Encyclopedia, which fills in entries on monsters based on how many of each the player has killed and requires us to keep track of kill counts of every enemy.

If your encyclopedia is stored in a set order you can use an array of characters or integers a use individual bits to represent whether each monster has been encountered. As for kill counts, just save out a series of integers in the same order as the monsters are listed in your encyclopedia.

I personally prefer binary files.

I needed a way to save the storyline state for one of my RPG games, and the method that I used was a single unsigned char value.

The storyline never had more than 255 possible states, so by saving say a value of 7 to the file, I knew that there were 3 storyline states.

In binary that would be 0 0 0 0 0 1 1 1 and you can see that there are 3 "on" flags.

This allowed me to hold quite a bit of save game data in a small space.

My inventory was stored as a single unsigned int value, in the same manner.
The first 16 bits of the value told me the items that were in inventory, and the
second 16 bits told me the quantities of each item.

There are a lot of fun tricks you can use for save games.

Avoid XML like the plague btw, that will slow your game down quite a bit, and XML loading code is a nightmare. True tinyxml makes it a little better, but its NOT good for this kind of thing.

Use XML for interoperability between tools.
Binary for games.

For instance I wrote an SVG conversion tool that let me use Inkscape to design game levels.
SVG is an XML based image format.
My tool extracted the data I needed to create full game levels quickly.

Game Programming is all about finding the solution to the problem at hand.
And boy is it a lot of fun! :D

Mybowlcut

Jul 3, 2009 at 3:32 AM

fwrite(guy1.get_name(), sizeof(const char*), 1, fp);

name = element->GetAttribute("name");

I know what I'd prefer...

As for speed, it runs at a fine speed with my game.

Zahlman

Jul 3, 2009 at 11:05 PM

Quote:
Original post by Mybowlcut
*** Source Snippet Removed ***
*** Source Snippet Removed ***I know what I'd prefer...

Those source snippets are entirely unrelated. You are conveniently ignoring the fact that data from an XML file needs to be loaded and parsed into some kind of structure before you can use the nice friendly interface of that structure.

Speedo_

Jul 3, 2009 at 11:26 PM

Quote:
You are conveniently ignoring the fact that data from an XML file needs to be loaded and parsed into some kind of structure before you can use the nice friendly interface of that structure.

Which is hardly an issue given the ten thousand or so XML parsers available for free.

I generally prefer binary files for saves, but I can't really see XML being a problem unless you're talking about truely massive save files.

josh1billion

Jul 3, 2009 at 11:30 PM

I agree with the idea of using binary files.

It can make debugging more difficult (since you can't as easily open a binary file with a text editor to check and make sure it's in the correct format as you can with a text file).

The advantages outweigh that, though: binary files can be written and read faster than text files (though this won't matter much unless your save files are extremely large or you're working on a slow platform, like a classic console). More importantly, as was pointed out by another poster, this will make it much more difficult to edit the save file and cheat, so most players won't know how and most of the rest won't bother. Lastly, you'll learn a lot about organization when you create the file structure. :)

In all actuality, it won't matter much which route you should. But you may as well use binary files since they do have some advantages over text files, and because you'll probably need to use binary files in bigger projects later, so why not go ahead and get some practice with them now?

Mybowlcut

Jul 3, 2009 at 11:42 PM

Quote:
Original post by Zahlman
Quote:
Original post by Mybowlcut
*** Source Snippet Removed ***
*** Source Snippet Removed ***I know what I'd prefer...

Those source snippets are entirely unrelated. You are conveniently ignoring the fact that data from an XML file needs to be loaded and parsed into some kind of structure before you can use the nice friendly interface of that structure.

Conveniently ignoring...

ticpp::Document xml_doc(file_name.file_string());xml_doc.LoadFile();ticpp::Element* root = xml_doc.FirstChildElement("blah");

You now have access to the whole file... still nice and friendly. :)

Matt_D

Jul 3, 2009 at 11:49 PM

Quote:
Original post by Mybowlcut
Quote:
Original post by Zahlman
Quote:
Original post by Mybowlcut
*** Source Snippet Removed ***
*** Source Snippet Removed ***I know what I'd prefer...

Those source snippets are entirely unrelated. You are conveniently ignoring the fact that data from an XML file needs to be loaded and parsed into some kind of structure before you can use the nice friendly interface of that structure.
Conveniently ignoring...*** Source Snippet Removed ***You now have access to the whole file... still nice and friendly. :)

but its still XML, and you still need to hit it using XML functions :P and you still need to *parse* the XML.

anyway, best bet is to start with XML, then once its working, move to a binary encrypted format.

a) its easier to test things out with XML
b) once you know it works, moving to binary is pretty simple.

this isnt a speed thing, but a protection thing.

DrEvil

Jul 4, 2009 at 12:15 AM

I agree. XML is a bloated and slow format. It's useful if you want to use a 3rd party parser and get something up and running quickly, but IMO there are better alternatives still to that. I personally prefer to use the scripting language(lua, angelscript, etc) that I typically always have linked into the game as well to store the value in a simple hierarchy of nested tables. It's human readable, easy to grab values from, iterate, etc. Duel use with the pre-existing scripting language. The tables can be pre-compiled and encrypted if you want, etc.

The down side is no schema validation and such that you could take advantage of if you used one of the huge xml loading libraries. Rarely in my experience is using that stuff worth the extra bloat, and it's pretty trivial to write scripts that walk your script table hierarchy to do some simple validation.

Mybowlcut

Jul 4, 2009 at 12:57 AM

Quote:
Original post by DrEvil
I agree. XML is a bloated and slow format. It's useful if you want to use a 3rd party parser and get something up and running quickly, but IMO there are better alternatives still to that. I personally prefer to use the scripting language(lua, angelscript, etc) that I typically always have linked into the game as well to store the value in a simple hierarchy of nested tables. It's human readable, easy to grab values from, iterate, etc. Duel use with the pre-existing scripting language. The tables can be pre-compiled and encrypted if you want, etc.

The down side is no schema validation and such that you could take advantage of if you used one of the huge xml loading libraries. Rarely in my experience is using that stuff worth the extra bloat, and it's pretty trivial to write scripts that walk your script table hierarchy to do some simple validation.

This sounds interesting. Could you post a snippet of one of your lua files so we can see what kind of structure your data is stored in?

DrEvil

Jul 4, 2009 at 1:08 AM

It's pretty simple, I essentially save the data out to a table, and the file itself is a global _MG table with each element underneath representing a 'goal'. What I like most about using a script based format is that it doesn't require any new libraries over what I'm already using, and it maintains some degree of type information. Loading the data is basically running the script, then I iterate the _MG table and for each sub table I grab the fields out.

I've recently begun using a 'schema' object that I have created very recently to reduce alot of the scripters need to do validation on their data. I won't bloat this post up with that though, you can read more about it here on the game monkey forums.

global _MG = {	CAMP_test2 = 	{		TeamAvailability = 4,		SerialNum = 34,		Stance = "crouch",		Roles = 0,		MinRadius = 32,		AimVectors = 		{			Vec3(0.611, 0.792, 0.006),			Vec3(0.991, -0.132, -0.016),			Vec3(0.137, -0.990, 0.017),		},		TagName = "test2",		Position = Vec3(283.106, 357.456, 0.125),		MinCampTime = 2,		GoalType = "CAMP",		Orientation = Vec3(-0.657, 0.006, -0.000),		Weapons = 		{		},		GroupName = "testgroup",		MaxCampTime = 6,		Version = 1,		Name = "CAMP_test2",		Radius = 64,	},	...

DrEvil

Jul 4, 2009 at 1:17 AM

If I were to use your example for what it would look like as a script output, it would be something like this.

TileCache ={    [1] = // i'd prob make the index the tile id rather than TileId=1    {        Name="thick_grass",        FileName="Tiles\thick_grass.png",        Animated=false,        ColourKeyed=false,    },    ...}

It's more lines than yours, but it's more readable IMO, since it's spread out and cleaner than the syntax that XML requires.

If the structure changed radically it would be trivial to write a script that went through the table and converted the data. Being a text format, the raw size isn't near as good as binary, but due to repeated use of certain keywords for fields, etc, it still compresses extremely well into an archive.

Mybowlcut

Jul 4, 2009 at 1:28 AM

I think I understand that... what does _MG stand for though? Is this in Lua?

As for readability, I agree that yours is easier to read, but my hope with my game was that I'd have tools (level editors, etc.) that would prevent the need for meddling with data like this by hand (not that it's happened - hard working on a game and a level editor at the same time).

sirGustav

Jul 4, 2009 at 1:51 AM

boost.serialization, you can use a xml or a text based format and easily switch to a binary, for the extra speed, when you release.

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.