Xml database schema

MySQL is fast and free.

it is probably bias due to the foolish round holes and square pegs i have to deal with at work, but XML is the devil.

Thanks.

I want something faster and more scalable. Maybe I'm wrong about the scalability? Does Mysql offer failover, and the ability to cluster (scale out) machines so that I can have more than 150,00 concurrent users (and under 5 millisecond response times for single record queries)?

Any special reason, why you like mysql? I like postgres, but I'm finding that it is too slow for my purpose. I just assumed that MySql would offer similar performance.

I also might want to embed it into my project (if necessary), without implicitly making my project open source. This is a closed source project. I want to make money off of it.

I can see how using xml in the wrong situation would make you feel that way. You might feel better if you try to think about xml like a configuration file with support for hierarchical data from a couple of libraries.

Microsoft tools make using xml very, very easy. I bet that the mono project on Linux duplicates that effort and makes it easy there too.

Does any have any more database advice?

I need something extremely fast, and lightweight.
Or, I need someone to point me to a database xml schema.

I'm no expert, but I'd have to guess that a text based XML type system would be slower than a slimmed down mySQL server.

MySQL is generally faster than postgresql because it supports less features, thus there is less overhead. Postgres is more of a database for any needs, while mySQL is focused on web content.

I'm not sure about features, but the new version of mySQL (5.0) is suposed to be really sexy.

Are you doing this in .NET by any chance? If you are the DataSet object has some really slick functions for working with XML.

Quote:

BradSnobar: I want to get an extremely fast, and extremely scalable database for my game.

Quote:

krez: MySQL is fast and free.

Quote:

BradSnobar: I want something faster and more scalable... [I want] failover, and the ability to cluster (scale out) machines so that I can have more than 150,00 concurrent users (and under 5 millisecond response times for single record queries)... ...I am willing to roll my own database system.

So, my question is do you want to write a game or write a database? The features you're asking for would be an incredible task just by themselves, hence the reason that databases that offer these features are so expensive.

The features you want are non-trivial. And the fact that you're looking for a shortcut to writing the schema -- likely to be one of the most trivial aspects of the an undertaking such as this -- indicates to me that you likely don't have the requisite expertise to create the software you're looking for. I know I certainly wouldn't.

I've worked with an internal team of professional engineers (roughly a dozen senior-level engineers) building an XML database that's scalable and version-controlled. Our requirements were far more flexible than what you suggest and we still had serious scalability issues.

I'm not wanting to rain on your parade, but I'd also hate to see someone dump a lot of effort into something that's likely to yield no dividends (or at least not get you any closer to what you're hoping to accomplish).

If you truly need enterprise-scalability (clusters, low latency, etc.) then you're best bet is to look into existing database technologies. If a database like MySQL or Postgre doesn't offer what you're looking for, then I'd suggest looking at how you can leverage those products to get the results you want.

If you're really wanting to use an XML database, then I'd suggest taking a look at Berkley SleepyCat.

Thanks for the feedback. Here are some comments in reply.

Quote:

So, my question is do you want to write a game or write a database? The features you're asking for would be an incredible task just by themselves, hence the reason that databases that offer these features are so expensive.

I agree, and I would rather not undertake this task unless absolutely necessary. Even in the most minimal way, I'm looking at six months to a year of work going in this direction, with additionaly features as time allows. (I do have the work experience with the various technologies necessary to get this done.)

Quote:

The features you want are non-trivial. And the fact that you're looking for a shortcut to writing the schema -- likely to be one of the most trivial aspects of the an undertaking such as this -- indicates to me that you likely don't have the requisite expertise to create the software you're looking for. I know I certainly wouldn't.

I wouldn't mind writing my own schema. I'm just looking to see if there are any industry standard specifications that I can follow for this. If I end up doing this, then I want to start from something that a standards body already agrees is a good idea. Besides, if there is a standard, then I can leverage other people's tools along the way too.

Quote:

I've worked with an internal team of professional engineers (roughly a dozen senior-level engineers) building an XML database that's scalable and version-controlled. Our requirements were far more flexible than what you suggest and we still had serious scalability issues.

This is extremely helpful knowledge for me. This would greatly effect my decision, as there are a couple of other options that I am pursuing too. Can you elaborate on this at all?

Quote:

I'm not wanting to rain on your parade, but I'd also hate to see someone dump a lot of effort into something that's likely to yield no dividends (or at least not get you any closer to what you're hoping to accomplish).

I don't want to waste effort either. The main reason for the post is that I wasted a bunch of effort creating a huge relational database in postgres, and it is far too slow. Even if I change the database engine to have some scale out functionality it would still be pretty weak in the performance department. So, now I'm looking at creating a proprietary solution that I can fine tune and scale accordingly. I figured while I'm at it, I can just implement it according to a standard if there is one.

Quote:

If you truly need enterprise-scalability (clusters, low latency, etc.) then you're best bet is to look into existing database technologies. If a database like MySQL or Postgre doesn't offer what you're looking for, then I'd suggest looking at how you can leverage those products to get the results you want.

I might end up needing to do this in the end, I'm just trying to explore options before I lock myself into a vendor like oracle or ms (they are the only ones that come close, so far) for more than 200k. I would sooner have my own less elegant, but much faster and more scalable solution if possible. This is mostly because I have a lot of time and skill, and not a lot of money to spend.

Quote:

If you're really wanting to use an XML database, then I'd suggest taking a look at Berkley SleepyCat.

I was looking at this earlier today. This is interesting. I might be able to write some tools to implement the scalability and help the time requirements while leveraging their tools too.

I know this is hard work and non-trivial. But it's a big problem that I need to get figured out before I move on with my project.

Since Mysql has been suggested a couple of times, has anyone ever personally clustered some mysql machines together? What has been your experience with it?

DrjonesDW3d
I am doing this on the .net platform. And yes the dataset xml stuff is pretty darn cool.