Resource File using XML

Hmm.......The XML protion didn''t come through what is the key word needed to make a post ignore markup?

[ source ] [ / source ]

---

RacingTREME updated.

ty, here is the example xml.....

The only problem I can think of is with binary data. for instance, it can directly contain "<" because then the xml parser would think it''s another tag. You can get around it easy enough, but it''s something to think about.

quote:

---

The only problem I can think of is with binary
data. for instance, it can directly contain "<"
because then the xml parser would think it''s
another tag. You can get around it easy enough,
but it''s something to think about.

---

You can get around this by Base64 encoding the data and putting it in quotes.

quote:

---

Original post by PaleRaider
I am about to begin writing my file resource system and was brainstorming a few ideas. I had the thought of using XML as the basis for my resoure file.

---

Why did you have that idea?

Personally, I''m using XML to organize everythng from Windows(directX/OpenGL),GUI(active, not passive) to saved games. Turning your back on XML is like asking for more work, imo. It has and keeps saving me time, making it possible to do things that would hinder my timeframe.

I had the idea because as I stated I was using it for all aspects of my engines configuration and wanted to try and have a standard way of retrieving and storing all data. I also like the inherent structure that is provided by XML, also by using a schema I can have type safety, not to mention the ton GUI widgets available that I wanted to use to create a resource editor.

Yet as I have found out using XML for a resource file system is a little bit overkill right now, especially when dealing with large amounts of data (over 200 megs). It takes way to much time to load using either DOM or SAX regardless of the parser (Xerces or MSXML, only good ones with schema support that are free). The other down side is DOM makes you load the ENTIRE file at start up, which is why it doesn''t work for large data sets. On the other hand SAX can end up creating more object in memory than DOM depending on how the handlers are written. Yet if the data resources needed for a game are small, less than 20 megs then it is a fairly good solution, it is easy to create the code to manipulate the data and very easy to create a nice gui to create the file.

In Java there is an API called jdom (www.jdom.org) that loads parts of an XML document as needed, which greatly reduces the memory footprint. If there were such an API wrapper for C++ then I wouldn''t have a problem because the load time could be very high, seeing that you only load what you need. Unfortunately I haven''t found one yet. So I basically spent a week writing and testing the system just to have it be unacceptably slow, oh well live and learn........

Please will you spend the 10 seconds it will take to rectify the formatting on your first post.

Base64 encoding it would be the way to go, but you wouldn''t necessarily need to enclose the data in quotes. You could also use a <![CDATA[ ]]> section

quote:

---

Original post by PaleRaider
I had the idea because as I stated I was using it for all aspects of my engines configuration and wanted to try and have a standard way of retrieving and storing all data.
[quote]
I also like the inherent structure that is provided by XML

---

XML does not have any inherent structure that is relevant to a specific problem. You still have to define the structure of an XML document. People are fooled into thinking that using XML means that they can take an off-the-shelf parser and that will parse their data for them. That is not true - once you have parsed out the parts of the document that are of interest to the application, you then have to go on and perform further parsing and also translate data into required internal formats. The only problem that an XML parser solves is the problem created by having a document written in XML format. Some people also believe there is intrinsic value to having their data in a human readable format. That might be so in some cases, but it is often indicative of a problem that you are even dealing directly with data. Just as with any data format, you should have tools which allow the user to work with the semantics conveyed within the data, not with the data itself. It is unlikely that a generic XML editor is going to provide the semantics you need for your own application domain.

quote:

---

also by using a schema I can have type safety

---

A schema has nothing to do with type-safety. The presence of a schema allows you to check structural integrity (at a high-price), but does not provide any guarantees about type. In fact, XML is not type aware. If you want type information, you have to define the structures which will convey that information, and you will have to write code to enforce the semantics of the structures.


[edited by - SabreMan on January 8, 2003 1:30:34 PM]

SabreMan,

It is true that XML doesn't have an inherent structure that is relevant to a specific problem. Yet I didn't state that it did. The basic structure of XML is that of a tree, with one root node. This is the basic structure I was speaking, I should have made myself more clear. You also stated that "You still have to define the structure of an XML document", I did that as well, the XML, I finally posted, was generated automatiaclly from a schema I defined for this particular case.

Even though I do like having my data in a human readable format, for the most part, I didn't state I would be editing it by hand. I planned on creating a resource editor that worked with my given set of resource files and classes. It makes no sense to create a system that is to store megabytes of information and edit it by hand, yuck. I also agree with you that that parser does nothing more than give you the nodes of the tree and validate it against a DTD or schema, the hardest part is making sense of the data. For this task that was trivial, the schema is small and the data in each discreet node is small as well.

I fear I perhaps worded part of my last post wrong. As you pointed out schema's don't in them selves provide for type safety. Yet they do provide meta-data about the type of an element or attriubte, of course you have to do decide the type when you create the schema. Yet if you combine this meta-data with an editor that is capable of enforcing the meta-data type you do get type safety, i.e. using the editor means you will not have invalid data types. That was another reason and benefit of creating the editor. Also you might want to check out the XML Schema Part 2: DataTypes specification. http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/
It describes in detail the types that are supported and their valid formats, including primitives.

Anyrate it is still a viable option as I said for small sets of data and configuration. Yet at this time is is a bit too slow and memory intensive for larger data sets.


[edited by - PaleRaider on January 8, 2003 1:58:36 PM]

SabreMan,

Using an XML schema you can in fact enforce type. The parsing routines I use in our application to load the GUI elements of the client(ala EverQuest) load a XML file, validate it with a schema and then parse the file accordingly. If the validation fails the procedure exits and parsing does not occur, instead returning the error from the validation routine. Validation will fail if a tag defined as :xsString or :xsInt is encountered and the data portion of the tag does not meet this requirement. XML Schemas were designed to do this and do it rather well.

Look up XML Schemas, they make the case for using XML much stronger. My application no longer has to validate a configuration file on its own(either for structure or for data quality). Using a Schema and the XML source, I let MSXML do it for me.

Kressilac

quote:

---

Original post by kressilac
Using an XML schema you can in fact enforce type.

---

I''ve just been reading this, which proves me wrong, so my apologies for the misinformation. By way of explanation, I was dealing with Schemas around 3 years ago, which was before datatypes were introduced.

quote:

---

XML Schemas were designed to do this and do it rather well.

---

That''s where you and I shall have to disagree. XML was not initially designed to carry such semantics, and has had something bolted-on as an addition. It''s a crude and ugly hack, and it also has clear problems, such as it only enables limited expressivity for describing a type-system, which railroads designers. The issue is that you have to be aware of something which solves the problem in a superior way before you will agree with that. Check out symbolic expressions. XML is OK in some contexts, but as with every buzzword, the case for XML is massively overstated, and some of the W3C recommendations are positively perverted.