XML?

I think I can give you one reason why XML is not used to create a widely used model format:
It''s text.
If I understand XML correctly, the proper design would require something like number just for one coordinate of a vertex of a triangle!
Just for that one coordinate (lets assume number is like 1.3) I used 20 characters to describe a coordinate. Then there are the y and z coordinates (+ 20 more characters apiece). Don''t forget vertex descriptors and color info, shaders, materials, and so on.
And also besides binary files being more compact to store, they are generally faster to read because they require minimal parsing of strings.

<FirstName>Elvis</FirstName>

One word: stored-size-versus-data-size.

If you reduce the size, you reduce the comprehension and thus you lose the whole point. I'd prefer a compact binary format any day, thankyaverymuch.

Later,
ZE.

//email me.//zealouselixir software.//msdn.//n00biez.//
miscellaneous links


[edited by - zealouselixir on January 4, 2003 9:53:31 PM]

[edited by - zealouselixir on January 5, 2003 2:38:08 PM]

You''re wrong buddy. XML just describes data , not necessarily providing data (have you ever looked at a skin file?). The model exporter could create files that are binary depending on what shader/texture/whatever those vertices are using. The files would not have or need a header, as it would be included in the XML file. That would make it just a straight binary dump. The XML would just link those vertices with a shader/texture/whatever and provide information about the file.

Let me give you an example:



[  c o d e  m a t r i x  ]

I''ve been using XML style data files for my games for quite a while now, and its really pretty handy (mainly because it makes hand-editing clearer and more foolproof). Models I keep in binary though - primarily because I don''t need to hand edit them.

A simplified texture definition file:



Note that its not ''pure'' xml since that there''s additional restrictions (newlines are important for example). But this is only because I write my own parser and so make things easier for myself.

I''m toying with the idea of xml logfiles as well, and a schema/stylesheet to make them display nicely in a browser window, which would be nice for highlighting error and important messages other that the tried and tested "!!! ERROR: x !!!" method..

It makes sense to use only formats that can easily be read and modified by anyone in game development. I''ve been using only plain text files (except for the actual graphics data) in my projects for some time now and it works like a charm. The fact that you don''t need to spend time on Custom-Format-Compiler alone is enough to justify the decision IMHO.
Using XML is only a logical extension to this.

If you don''t like the idea of having thousands of files floating around, you can always put them into a .zip file or similar.

I don''t know whether there''s a standardized XML model/shader format yet or what it should look like, I''m just defending the general direction (although the fact that John Carmack goes into this direction should be enough for most ... )

cu,
Prefect

So...how would you write an XML parser? wouldn't it be much easier just to have

Grass Grass.png No

instead of...

>Texture name="Grass" filename="grass.png" bumpmap="no" /<

??

[edited by - SuperRoy on January 5, 2003 12:24:23 PM]

You dont have to write your own XML parser because there are tons available for free on the net.

"wouldn't it be much easier just to have

Grass Grass.png No

instead of..."

Easier maybe, but not if something needed extending. As-is, the contents of tags are read in as key-value pairs (eg. name="Grass") which makes it easy to extend - bumpmap wasn't originally in there for example, but easy to add. Also, xml's nested structure lets you add a hirarcy easily: I'm only using a single texture group at the moment, but adding tilesets etc. would be pretty easy.

As for the parser, it basically has a set of states, and switches between them depending on the open/close tags, ignoring anything it doesn't recognise. It won't handle complex xml, but there are libs avalible if you don't write your own.

[edited by - OrangyTang on January 5, 2003 12:33:43 PM]

You can embed binary data in XML. This is similar to the way you can embed images in HTML, although the XML equivilent is more powerful.

For example, Visual Studio .NET uses XML to create .resx resource files. When creating a C# application, the IDE automaticly embeds graphics in the resource file. However it is maintained as XML so that humans can read it and tweak it. It also facilitates the easy manipulation by an outside program by easily exposing the definition for the file format. The schema that defiens this format is either embeded in, or referenced by, the XML file.

This is also great for creating a globalizable format. It can be used as a link between two or more other formats (see SOAP). Through the use of schemas, a enforced definition for the format exists. This can be used to make the format easily understood by a wide range of programs and people. Insert some comments into the scemea that defines the binary aspects of the format, and it could be real simple to transport data (provided that those comments are rules that are abided by and are kept up to date because only the XML schema can be automaticly validated, the binary data must be assumed to be correct).

I plan on using XML to store metadata with my textures, such as their resistance to bullet passthroughs and walked-on sounds.



-SniperBoB-

What if The Matrix was recursive?

quote:

---

Original post by SnprBoB86
This is similar to the way you can embed images in HTML,

---

You can embed images in HTML?? Since when? How?

---

I''ve looked at the source, and there are pieces that are good and pieces that are not ... My experience and some of my friends'' experience is that Linux is quite unreliable. Microsoft is really unreliable but Linux is worse. -- Ken Thompson

I don''t think that XML can embed binary data. HTML does not embed images, it just links to them and the browser renders them. That is not unlike what I am proposing. You can use an XML parser, and use something like paks to store data. Simple as that. I don''t see how some people can not like it. It is what XML was made for.

[  c o d e  m a t r i x  ]

My example stands. If you want other people to modify it, write a fucking tool, but don''t make me format my data (or even my "data descriptions", whatever the hell they do) like HTML and blow its size to 3x the norm and 10x the binary form.

Later,
ZE.

//email me.//zealouselixir software.//msdn.//n00biez.//
miscellaneous links

quote:

---

Original post by elendil67
I don''t think that XML can embed binary data.

---

Sure you can:

---

Update GameDev.net system time campaign: ''date ddmmHHMMYYYY''

After doing a little research I've found some stuff out.

First off, ATI Rendermonkey stores its platform-independent shader data in an XML database, so there is a precedent for shaders. As for models, I don't think there are any existing XML model formats, and this is probably because of the size limitations mentioned earlier. It's thoroughly infeasible to actually encode each vertex's data in text, even with some rounding, because it gets inflated several times over the binary format, and you end up with like 100 MB models and stuff. The only reasonable solution is to use binary. The immediate problem at hand is that you cannot *directly* put binary data into an XML document because it could interfere with the parsing. However, you CAN (as SnprBoB86 and dalleboy pointed out) use certain encodings to store it indirectly, such as base64. Base64 is still a little bit bigger (33%) than normal binary data because it encodes 3 'unencoded' bytes in 4 'encoded' characters. However, base64 was designed to store MIME data sent in email, where the set of restricted characters is MUCH bigger (there's a lot of 7bit stuff going on, I think) - whereas in XML it's basically safe to use everything outside of a couple special characters like &, <, >, ", and '. (Note that you can also encode the whole document in 16-bit as well, but we're assuming we're using UTF-8 here.) Therefore you *can* create an encoding method for storing data in UTF-8 encoded XML document that would be much more efficient than base64 (perhaps < 10% bigger than the original data.) The best way to go about this would be to find something that's already standard somewhere else, but I'm not sure what's out there. In addition, vertex data compresses very well, so you could end up with it gzipped then encoded in the XML document and take up LESS space than in just binary format, which seems like a possibility. In order to keep it flexible you'd need to describe the method in which the vertex data is encoded in the XML document, and you'd end up having to write a pretty darn flexible model loader, but it's definitely feasible. Just some facts and thoughts.

[edited by - SHilbert on January 5, 2003 3:17:03 PM]

You may be suprised, but XML is actually more widely used in game development than you think. To name a few:

- Mathengine''s Karma (Unreal2k3) uses XML to describe the physical model of objects

- Criterion''s RenderWare Studio uses XML to describe the game database (pretty common)

- Criterion''s RenderWare Graphics 3.5 will use XML to store an intermediate scenegraph representation of exported data from the art packages. This data is later compiled to binary for release.

- 3ds Max exports its geometry to text (though not XML) if you wish. Maya does as well.

- Metrowerk''s Codewarrior uses an XML description to describe its user-interface extensions.

I love XML. It''s a hierarchical data structure, easily traversible from code, widely parsable in various tools, and very human friendly.

quote:

---

Original post by elendil67
Obviously it pertains to graphics programming. Anything? No? Model formats. Shaders.

---

Isn''t VRML already an "XML based" scene format ?

I use XML only for human tweakable stuff, sure it''s ''big'' (well not that much seeing available disk space), but humans can edit it with notepad or anything else.

XSI offers an ASCII and a Binary format, although it''s not XML, the ASCII format is easily understandable.
The binary format is much smaller...

For anyone thinking about allowing the game to be downloaded, the smallest the best, for CD distribution, you don''t care that much.


-* So many things to do, so little time to spend. *-

quote:

---

Original post by max621
I think I can give you one reason why XML is not used to create a widely used model format:
It''s text.

---

However, Doom III''s models are stored as text. It may not be widely used currently, but when games using that engine are released, itll be more and more common.

-----------------------
"When I have a problem on an Nvidia, I assume that it is my fault. With anyone else''s drivers, I assume it is their fault" - John Carmack

Cinema4D exports to xml, too.. quite handy..

and compressed xml is awesome small.. most zippers have an extra text compression algo wich is quite efficient..

i don''t know why file size got mentoyed anyways.. anyone has problems with filesizes? in zips, they are small (so download is small), on hd, they are small enough (i mean.. half a mega is one chunk on the hd anyways.. or you can specify your folder to be compressed, and you gain nearly zipsize again:D)

longer loading time due parsing is the only problem i can see.. loading huge data models could get much slower due that.. on the other side, this gets less of a topic as well, as todays cpu''s are very fast compared to data access anyways..

and they are superb to edit, yep..

"take a look around" - limp bizkit
www.google.com