Saving/Loading objects in C++?

Started by
5 comments, last by Krohm 3 years, 5 months ago

How do you guys go about saving/loading (serialization/deserialization) objects in C++?

Advertisement

That's a tricky question because it depends on the level of loading you want to achieve. There is a difference in between loading “any” object from a serialized data source or just loading “known” objects at runtime. If you want to load nearly any object that might be possible in C++ then you need to overcome with some kind of RTTI/ Reflection system like what C# has. First you need to be able to determine the object type (via an ID) and load it's RTTI to be able to iterate and call the propper constructor method, while on a known object, the ID leads to some callback/ delegate that already contains code to properly assemble the object from it's binary source.

However, then there might be some methods involved that iterate over fields and properties to detect their type and reassemble step by step to the object instance that was saved to binary once.

I have done all of these recently while "knowing" was involved more in the sense of save files for games, a full fledged serializer/ deserializer has been used primary for tooling and editor code.

One implementation I currently use is half-half of both, some reflection like code is walking through a type on a precompilation step and locates those members that have a certain “attribute” attached to them (a comment that is formed like an Attribute in C# .e.g. //[Serializeable]) and collects those information for later use by a code generator. The code generator creates a function that is supposed to serialize the object instance into binary data and reassemble it from such data. The generator also adds property IDs to be able to have some kind of backwards compatibility to classes of a different version. They are simply defined in code (as an attribute) by the user and contain some header bytes that allow the assembly function to jump over the block of data if needed.

In code, I register certain types to their serializer and assembly functions on initialization. To get a clear type identification, I use a compiler macro that generates a type-ID for nearly every type possible in C++. The macro utilizes the __PRETTY_FUNCTION__ (and similar macros depending on the compiler) and some pointer shifting to get a more or less unifed name out of every type you can pass into a template function. The platform bound type-ID is then a 32 bit hash value of the string. But I also have some fixed IDs in code for types I need to be platform idependent (primitves for example) so it is a bit of both.

It works quite well at the moment and is fast in development as also production code

You can use Boost or Qt. I use Qt. Both contain helpers for serialization/deserialization.

Read this article: https://www.qt.io/blog/2018/05/31/serialization-in-and-with-qt

Watch these tutorials:

I've seen systems that used a custom binary serialization format with read/write methods on the objects directly, reading/writing the entire game state directly to/from disk without any pre-processing or structure, data structures that could be pointed at your specific object that the serialization code would then read, code generators that used an interface definition language to generate the serialization functions, code generators that parsed the code directly, entirely separate systems that read subsets of the objects and compressed them for network transport, and hybrids of one or more of these things. And then there's all the libraries out there that do one or more of these things for you, like flatbuffers and protocolbuf and so on.

It's hard to give advice without knowing what your specific situation is, because each of these have their own use cases and problems.

  • What are you trying to serialize?
  • Where is the data going/coming from?
  • How important is performance?
  • What concessions are you willing to make about how the objects can be laid out in memory and what types they can contain?
  • How much coupling are you willing to have between the objects and the serialization code?

Adding to Oberon_Command's good questions:

  • Are you reading and writing external files (which could be edited independently from your C++ data structures) or saving and loading your own objects (which are expected to be reconstructed exactly)?
    One of the two representations is the master, the other allows major shortcuts (like actually constructing detailed objects only for the parts of a file you are actually using, or altering file formats because compatibility with old files doesn't matter).
  • Do you want the files to be easily editable (e.g. JSON) or opaque (e.g. memory dumps with metadata and swizzled pointers) or deliberately hard to modify (e.g. with compression, encryption and checksums)?
  • Do you need the files to be portable between different platforms (i.e. different endianness, data type sizes, memory alignment that need to be handled in the loading and saving code)? Typical use case: load the same assets from, say, Windows 32 bit and Windows 64 bit builds of the game.
  • Do you need the files to be portable between different versions of your data structures (requiring versioning and separate code to load or save each supported variant)? Typical use case: load an old saved game (which the player has spent a lot of time and effort on) after an upgrade.

Omae Wa Mou Shindeiru

In the past, I have used mainly 2 methods.

  1. explicit pack/unpack methods. Very time consuming and prone to errors. Cross-referencing was a nightmare.
  2. Google protocol buffers. I'd stay away from them now, as far as I can tell they are effectively superseded by flatbuffers.

Fasten your seatbelts. Option 3: don't use C++. At some point I had a crazy VM-like thing which could just pull up everything transparently. OFC didn't blend well with hardware-backed buffers and such but for gameplay data, it was kinda awesome. This brings us to the real point: if you're looking for gameplay-specific data consider maybe you could consider using a higher-level scripting language. Right now it might sound nonsensical but if you can defer this whole save/load thing to the future it might make sense.

TL;DR: be specific to your needs. If being specific doesn't help, try again in the future.

Previously "Krohm"

This topic is closed to new replies.

Advertisement