Skip to main content
GameDev.net gamedev.net
🔒 Locked

Serializing without breaking encapsulation

Started by Ezbez Jul 23, 2007 at 4:30 PM 17 replies 4.2k views
Original Post
Ezbez
Ezbez
Lately I've been working on using Python's pickling system with my game. I plan on using it to make save files of my game. I'm wrapping C++ classes into Python via Boost::Python. The problem comes when I write the pickle suite class which forms the tuples of information that need to be saved and then translates that information back into the class. This pickler class requires that I know about some of the internals of the class. Usually it's nothing vital - much of it was already part of the public interface. But sometimes it starts to get nastier and nastier exposing things that I had no previous need to expose, even if it doesn't hurt to do so. It feels messy, like a hack. I don't like it. Since boost::python (which I use) basically mandates that you have a separate class so that it can inherit from pickle_suite, I find it forces me to expose this stuff to the whole program. Ick. The obvious solution to me is friend classes, but I'm not so sure that they're a good solution. What is a good solution? On a side note, there are some things that I'm pickling which are part of another library. Some of these things do not give you all the information that you need to completely reconstruct the class nicely. For example, my physics engine's solid class takes in some shapes when you construct it, but there's no way to get those shapes back after it's constructed. It has redundant information and is a little annoying to write, but it works. I've gotten around this by writing a wrapper class which stores the shapes as they are passed in. Again, what is a good solution for this?
Vorpy
Vorpy
Writing your own wrappers to store information needed to recreate objects that don't give access to their internals sounds like a good solution, and if the shape information is already being pulled from some data source like a file then to recreate it you just need to go back to that same data.

C++ wasn't designed for serializing objects, so I think anything that does serialize them is going to look like a hack in comparison to the serialization available in some more modern languages. You could give a class public methods that aren't exposed to python that are used by your serializing class to save and load an instance's state.
Aressera
Aressera
Impossible, that's one of the reasons why people don't use serialization in real products. Serialization is dependent on implementation. Therefore, if you write a class to disk for version 1.0, then change that class even slightly in version 1.1, you won't be able to resurrect that old data. In order to do this you need to use a method to separate the file representation of a class from it's implementation.

The standard way to do this is with an additional class called a gateway - it takes a class and write/reads it to/from the disk/database/whatever data storage mechanism. That way you can keep all of the reading/writing code in one place, making it easier to deal with version changes and also keeping your implementation hidden.
T1Oracle
T1Oracle
One good solution is to create a separate class that reads/writes the object to/from a data source. That way changes to the class do not require changes to the data format, and they may not even require changes to the reader/writer class(es).
Programming since 1995.
Aressera
Aressera
Quote:
Original post by T1Oracle
One good solution is to create a separate class that reads/writes the object to/from a data source. That way changes to the class do not require changes to the data format, and they may not even require changes to the reader/writer class(es).


i just said that
Ezbez
Ezbez
@Vorpy - Unfortunately, I can't just go back to the file they are loaded from (this *is* the file they are loaded from). :( But I did just find out that my physics engine has "GetData" and "SetData" routines which return a simple struct that contains all the information to store and restore them, so that solves that particular problem. Speaking of which, is this a good way to go about it? To have GetData and SetData routines that fill out a basic struct and then can return themselves to that state.

@Aressera - I don't see how serializing something is any different from any other method of saving and or loading files. So I guess I'm saying that I don't really know what you're getting at with your first paragraph.

@Aressera's second paragraph and T1Oracle - I think I already have what you're talking about. Python reads and writes it to and from the disk for me. I don't know any of the details about that operation. I'm worried about the process of getting information from a class into Python's pickling system without breaking the encapsulation of said class. ASCII diagram time!
*---------*         *----------------*          *--------** MyClass * ------> * MyClassPickler * -------> * pickle **---------*         *----------------*          *--------*              ^^              ||    Here's the encapsulation problem.    MyClassPickler needs to know    about MyClass And I find    that icky.

MyClass - some class that I want to save and then load from a file
MyClassPickler - turns the current state of a MyClass into a tuple, and can use a tuple to return MyClass back to a state
pickle - Python's pickling system. It takes the tuple and writes it to a file, and can then load from the file and turn it back into a tuple.
Gorg
Gorg
Quote:
Original post by Ezbez
*---------*         *----------------*          *--------** MyClass * ------> * MyClassPickler * -------> * pickle **---------*         *----------------*          *--------*              ^^              ||    Here's the encapsulation problem.    MyClassPickler needs to know    about MyClass And I find    that icky.



You should not feel icky. You have to see those 2 classes as working together. They form a system, an interface to work with the data in MyClass.

To do this cleanly in C++, you don't even have to create new accessors or use friend, you can simply nest the classes.

class MyClass{public:     class MyClassPickler     {     }}


You get free access and a clear notion that they are strongly related.

If you don't want to do that or use friend, you can have MyClass emit a state object(could simply be a struct with a copy of all the data) that contains all data necessary to be saved and have MyClassPickler work on it. MyClass then needs another function to reload from the state.
Aressera
Aressera
Quote:
Original post by Ezbez
@Aressera - I don't see how serializing something is any different from any other method of saving and or loading files. So I guess I'm saying that I don't really know what you're getting at with your first paragraph.


You probably just don't understand serialization. Serialization is a method of putting a class in memory into a representation that can be written to disk. This memory representation is totally dependent on the implementation of the class and totally breaks incapsulation. From the serialized class, you can tell what its fields are and what methods it has, whether or not they are public or private has no affect on the visibility. Therefore, if you decide to change the internal implementation of a class (private members) later on, you will be unable to resurrect the older versions that have already been serialized. Basically, if the way a class is arranged in memory changes at all, your program will break.

That is why you use a "gateway" object that uses data abstractions as well as getters/setters to write a format-independet version of that class to disk. Then when you go to ressurect a class, it doesn't matter what the actual structure of the class is, just as long as its abstracted data matches.
Aressera
Aressera
read this:
http://en.wikipedia.org/wiki/Serialization
Ezbez
Ezbez
Thank you, I think I see what you mean now. I will read up on nested classes, something which I have never used before.
T1Oracle
T1Oracle
A writer class will need access to the data members of the class being written. There is no way around that. The reader class may be able to just use the object constructor as an interface so that will not have to break encapsulation.

Regardless, I know of a few general ways of getting the writer class to interface with the object being written.

1) public data members - I hate this solution as it exposes everything to everyone. This is usually a bad solution, but occasionally it is ok.
2) friendship - I dislike this solution as well, as friendship means that the definition of the object must know about all of the writer classes.
3) Visitor pattern + passing data members - This method can get messy but it does preserve encapsulation. The general way in which I implement this is as:
writerObject writer;object.VisitMe(writer);/// VisitMe definitionVisitMe(writerObjectBase &writer) // you can use a common base class to allow for easy extensibility{   writer.Write(dataMember1,dataMember2,dataMember3);   // OR you can further encapsulate all data members into 1 object and do this   writer.Write(dataMembers);}

4) Include methods in your object that fully define all the information needed to serialize the object. - For an object as simple as a math Vector this solution works, but if doing this pollutes the user interface then it is a bad solution. If these methods are only used during serialization then it's likely a poor design decision.
Programming since 1995.
Ezbez
Ezbez
I read up on nested classes, but I don't see how they help. Either they have no way to access the internals of the outer class, or they must be a friend class. I don't see what benefits over a non-nested class they provide for this.

Of your solutions, T1Oracle, I am leaning towards number 4. Why do you say that it is a bad choice if all I'm using it for is serialization (which will be all)?
DanielH
DanielH
Why don't you give your serializable classes a serialize and unserialize methods?

class MyClass : public ParentClass{private:  float m_float;  int m_int;public:void Serialize(std::ostream& stream){  ParentClass::Serialize(stream);  stream << m_float;  stream << m_int;}void Unserialize(std::istream& stream){  ParentClass::Unserialize(stream);  stream >> m_float;  stream >> m_int;}};


Edit: Or even better, overloading the << and >> operators for the serialization :)
T1Oracle
T1Oracle
Quote:
Original post by Ezbez
I read up on nested classes, but I don't see how they help. Either they have no way to access the internals of the outer class, or they must be a friend class. I don't see what benefits over a non-nested class they provide for this.

Of your solutions, T1Oracle, I am leaning towards number 4. Why do you say that it is a bad choice if all I'm using it for is serialization (which will be all)?

Because it pollutes the interface. It exposes features that should not be used for anything other than serialization. If it is used elsewhere you end up tightly coupling other parts of your code when only serialization needed that tight coupling.

Now, if those methods are already a necessary part of the class design then it may be the solution.

Think about how the cost of changes to different features factors in to this.

Perhaps you should read about the Interface Segregation Principal. I borrowed that link from another good and relevant thread.
Programming since 1995.
T1Oracle
T1Oracle
Quote:
Original post by DanielH
Why don't you give your serializable classes a serialize and unserialize methods?

*** Source Snippet Removed ***

Edit: Or even better, overloading the << and >> operators for the serialization :)


I don't do that with binary. I remember trying it to load data in it didn't work until I reverted to using stream.read.
Programming since 1995.
Gorg
Gorg
Quote:
Original post by Ezbez
I read up on nested classes, but I don't see how they help. Either they have no way to access the internals of the outer class...


I should have added more details. It is supported in next version of the C++ standard. And it has been supported, as far as I am aware, in GCC since version 3.2 and since visual studio 7.1. Since it is easy to implement, they are just ahead of their time.

http://anubis.dkuug.dk/jtc1/sc22/wg21/docs/cwg_defects.html#45

So you do have access now.
Ezbez
Ezbez
Thank you for the replies. I will be sorting through them in a while, as I won't have access to a computer for the next three weeks. Meanwhile, I will think about what has been said.
wood_brian
wood_brian

Our approach doesn't break encapsulation or use
friends. We generate Send and Receive functions and
you add the prototypes to your class. The results
we've gotten have been significantly better than other
approaches performance-wise.

A recent thread on clc++m discussed a weakness in current
serialization approaches. The Boost library can be easily
fooled into allocating large amounts of memory in an
attack. This is currently true of our approach as well,
but we have acknowledged it and are working on a fix.
The Boost author has not acknowledged the problem as of
yet. If you want to read that thread search for denial
of service and serialization.

Brian Wood
Ebenezer Enterprises
www.webebenezer.net

Topic Locked

This topic has been locked by a moderator. New replies are not allowed.

Sign in to reply to this topic.