MMO Database

4. Large number of players (well say 100,000)

Wow, 100,000 players, really? That's impressive. How many of them will be playing simultaneously? Because, you might have to implement some sort of sharding or similar load balancing system, and mirroring databases across all the nodes is an art form in itself.

But anyway, many online games use several different types of databases for storing persistent data. For example, if you have data that naturally fits in a table with the same columns then a good ol' SQL database is probably the best choice, eg. postgreSQL is known to be reliable and performant. For data where the structure varies more some sort of no-SQL database might be a better option, eg. mongoDB.

Also note that you will most likely have lots of logging and auditing data generated by the game, ie. data that is not strictly speaking required for the game to run but allows you to more easily debug and develop the game. This kind of data is usually stored in a different database than the actual game state data.

I'm all in MySQL, it has a number of different engines: The most common are INNODB and MYISAM (I can't recall which one was what) and the difference is huge (and you can choose which table uses what). Both of which are implementations of the "new ways" and the "old ways" (I hate this misnomer).Transactional and atomic queries are the nice thing about the "old ways"... the "new ways" can corrupt data if done incorrectly (well, maybe it's unavoidable).
The thing is that you talk about 100k users. And a MMO. While making tests you will realize that you will step over unless you really know each step towards making that... and it might take a lifetime.

Specifically, I have trying to sort out what type of database would be appropriate.

The fact that you don't know is a good sign you are not ready for a game at this scale.

I am guessing the type of data base is one of the smaller problems that you have. In the end, all data bases are mostly the same, they store large quantities of records with data.

While there are no doubt performance differences at full scale, it won't make any difference now.

Just pick one. It's likely to be wrong, but replacing it with something better shouldn't be a major problem, as their interface is all similar.

Have you done any studies on what happens if you have 100,000 live network connections? As in, how much traffic does that give, at the computer side, how many network packets / second do you have to handle, how many of those need data base access, so how much time do you have for one DB query? How does that requirement match with whatever data base you have?

At the network side, how many network packets does that cause, how fast must your network connection be, what does that cost?

Inside the data base, how many records will you have, how fast does performance go down as the number of records goes up, etc?

Sort of a "big data stream picture" to get an idea how much iron you'll need for that scale.

https://www.gamedev.net/blog/355/entry-2250155-why-you-shouldnt-be-making-an-mmo/

Not really relevant for the OP here. He isn't talking about the traditional kind of MMO such as GW that is mentioned in the article but a Mobile MMO which is less of an MMO game and more of just a multi user database with a pretty GUI. Turn based, very limited amount of content. No need for a massive development team. Fairly small amount of assets, No real user customization, Networking is usually just a REST API over HTTP. There are literally hundreds of these being released on iOS and Android every week.


Sure 100,000 users is a bit ambitious but, with modern tools and cloud infrastructure its fairly easy to optimise and scale up when needed. OP could probably pull of a MVP just using Firebase or Parse Server.

1. No character walking around a large world. 2. The world for a player consists of a limited view of their game. Think Clash of Clans. 3. Game objects have varying levels of progression. 4. Large number of players (well say 100,000) 5. Inventory of a mixture of roughly 200 items 6. Many time based events

OP could probably pull of a MVP just using Firebase or Parse Server.

Sorry, but as this is For Beginners, adding some realism to the dream is (unfortunately) probably the best idea. The journey can be fruitful, and the person can learn quite a lot attempting to reach that goal, but encouraging an unrealistic goal isn't very kind.

While part of the question is about databases, the entire situation is entirely unrealistic. Even using great tools, a game at the scale being described cannot reasonably be implemented by a single human, particularly a beginning developer.

It is important to understand that the dream cannot be achieved by one-self nor with a small group of beginner friends. A large team of experts can accomplish it, certainly, but that isn't what they have.

Helping to redirect the goal into something more realistic can be fruitful, and that's what a few people have done.

But getting back to the original question:

I have trying to sort out what type of database would be appropriate. I have read posts and opinions on this topic ad nauseam.

Deciding what tools and technology to use -- including databases -- is mostly an exercise in elimination.

Make a list of all the possible software you want to use. Eliminate those outside your cost range. Eliminate those that have known problems you cannot work around. Eliminate those you know or believe you cannot work with, although leave them if your choices are limited and you can potentially work around the issues.

When you are done eliminating the ones that will not work, you should have a pool of those that might work. Pick one, either at random or because you have familiarity with it already.

If you ever discover the tool won't work, add that to your criteria, eliminate all those that fail the new test, and pick from among the contenders again.

At this point you have quite a few excellent free database systems to choose from. Study them a little bit, study your needs, and pick one. Pick from the list at random if you must.

While part of the question is about databases, the entire situation is entirely unrealistic. Even using great tools, a game at the scale being described cannot reasonably be implemented by a single human, particularly a beginning developer. It is important to understand that the dream cannot be achieved by one-self nor with a small group of beginner friends. A large team of experts can accomplish it, certainly, but that isn't what they have.

There are several MMOS that have been developed by one man teams recently:

Agar.io

Slither.io

Sherwood Dungeon

Love

Minecraft

Some of the developers probably have more experience than others (Notch obviously). Others though whilst having experience in software development in general, do seem to beginners at games development.

Besides the OP never indicated weather he was developing by himself or with others.

I have been working on a mobile MMO type game. Specifically, I have trying to sort out what type of database would be appropriate. I have read posts and opinions on this topic ad nauseam. The main issue I am having is that most are working on games that are not in the same vein and thus I cannot tell how relevant the opinions would be to me. So here are the details of my project.

1. No character walking around a large world.

2. The world for a player consists of a limited view of their game. Think Clash of Clans.

3. Game objects have varying levels of progression.

4. Large number of players (well say 100,000. Notice: I DON'T MEAN CONCURRENT)

5. Inventory of a mixture of roughly 200 items

6. Many time based events

Thoughts?

None of that tells us anything about what you want to actually store. Therefore there's nothing we can suggest regarding which database to use, as it's not even clear that you need one.

Lacking any further information, the best answer is probably Postgresql.

If you have an idea of what needs to actually be persisted to the database, based on your game's requirements, we can come up with a more directed solution (which may still be Postgresql, but anyway).

There are several MMOS that have been developed by one man teams recently: Agar.io, Slither.io, Sherwood Dungeon, Love, Minecraft

I hate derailing the topic this much, but here goes a little:

That first "M" in "Massively Multilpayer Online" means something. Multiplayer games can commonly handle a few hundred players. If interactions are limited they can sometimes handle into the thousands of players. The term "massively multiplayer", was originally used for games with over 100,000 concurrent players, and inside the industry generally means even more than that. An online game with hundreds or even thousands of players is not "massive". Even a multiplayer game with hundreds or thousands is not "massive". When it approaches or exceeds six figures of concurrent players -- not in lobbies that go to small games, but actually large groups together -- then it becomes an MMO.

Slither.io. Started as a one-man gig. A game could handle up to about 200 concurrent players before they became unplayable. The one-man developer became a multi-developer team, rewriting the servers to handle 500 concurrent players. MO, but not MMO.

Agar.io. The first version was by one individual. Then he started working with Miniclip, rebuilding the game and expanding it considerably. Even so, a map can have 128 concurrent players, so hardly MMO. Yes they have a high total player count, but they are all independent from each other and certainly not in the hundreds of thousands in the same world. Many thousands in the lobby, but that's different and an enormous number of products can boast of that feat. That is branding and player count in parallel multiplayer games, not concurrent players together in a massively multiplayer environment.

Sherwood Dungeon. Like the others, started out as a small online game by a single developer, and based on blogs and the wikia site his original system did scale rather well to about 4000 concurrent players. He was also an industry veteran. Looking it up online, he started around 2004 as a hobby, and slowly started growing it by himself. He quit his day job in 2006. By 2008 there was another worker, his wife. Soon there were "other respective parties", and everything shifting from "I" to "we" around 2010. Early 2015 he announced plans for a mobile edition that still doesn't exist, and a year ago the developer said he's getting a job as a regular programmer in another game studio. So it was a large-sized multiplayer online game, but never approached massively multiplayer scale.

And Minecraft. Notch made the first versions by himself, and if you played it from 2009 to 2010 it was by himself. If you joined in during the initial Alpha version you were looking at a roughly 10-person team. The credits listed in 2011, the "prerelease 1.9" version had 16 people. By the time they reached "beta" they'd hired and brought in double that many people again, and various people at the rapidly-growing company said they nearly rewrote the entire rendering system and most of the networking system, and they completely replaced everything on the server side. It had grown to roughly a hundred when Microsoft bought it in 2014, and with all the ports the total developer count is much more. The Minecraft you have played for the past four years had over a thousand work-years on it, far more than a single developer could do in ten lifetimes.

YES, an individual developer can build a game in the spirit of something larger, and with the goals of it becoming large. That is encouraged. Over in the Multiplayer and Network Forum FAQ are some documents where various people -- including that forum's moderator -- build MOG's quickly, including a small game server built in about four hours of work in Python. It could likely handle several hundred players, maybe even a few thousand concurrent players. A few people leveraged similar systems to put together full-fledged MOG's in about a week, no problem if you keep it minimal and know what you are doing. There have been Multiplayer Online Games since the 1970s, then called MUD (Multi-User Dungeon) and MUSH (a more social variant) games, and they are entirely within a single developer's scope.

NO, an individual developer does not have the capacity to build an MMO, in the real meaning of Massively Multiplayer Online Game, because even the most powerful off-the-shelf tools cannot handle it without a large team of experts and an enormous pool of money.

And trying to stay on topic as best we can, for a regular online game, the data that would be placed in a database can likely use any database system. Some designs may lend themselves slightly toward different solutions, but any of them should work in general. If a person has a specific reason to exclude one it should be excluded, but likely any of them would work.

In an effort to steer the thread back on topic... The choice of SQL-like vs NoSQL database typically has very little to do with the size or quantity of data, and a lot to do with whether or not you need transactional semantics.

In essence whether you need transactions boils down to whether it is important that everyone can be guaranteed to agree on the values in the database at all times. For a global item attribute that is only updated when you release a new version, the answer is most likely no. When two players are actively trading items/gold, the answer is definitely yes (i.e. if one player unplugs their ethernet cable in the middle of the trade, is it important to know who ends up with the items?).

In a practical modern setting, you will likely want three types of database-like systems in your game:

• A cheap, high-performance NoSQL database to store shared data like item attributes and descriptions, location data, etc.
• A transactional database (likely SQL-like) to store player gold, inventories, and maybe things like quest progress.
• And an in-memory caching layer to sit in front of the above and improve read-throughput (i.e. redis or memcached).

(there is also the question of whether you need rich queries/joins, but that's probably less relevant to that game itself than it is to metrics and analytics used to analyse the game's performance and operation)

@swiftcoder: Having 2 data bases, with memory caching, all connected together, is something we should recommend to a beginner? Say, right after "hello world" and MyFirstClass()?

@bluebass44: Please don't silently edit the first post, it gets missed. Also, it is hard to track what exactly you changed. Better post an update instead.

As for the question, I very much don't agree with your first assumption, but if you like close contact with big walls, nothing beats your own experiments.

Stuff is sufficiently complicated to throw random surprises at you, and you cannot select a data base based on answers from a single post of 10 globally descriptive sentences. The answer is try things. Get list of candidates, build an experiment for each of them, and throw a realistic load at it, and watch what happens.

For extra fun, build the network layer too.

Those things are sort of the core of the application, so that has to survive a huge load.

Given the stealth update to the OP, I'll throw in some more data:

1. Many game objects that have varying levels of progression - not every player possesses the same objects (This is what makes me begin to wonder about NoSQL).

There's nothing intrinsic to NoSQL here. This is fine for a relational DB. You'll probably have item data in standard DB tables.

2. Inventory of a mixture of roughly 200 items and associated quantities.

If these are instances of the items above, that data might also be suited to a relational database. Or maybe it could be a document in a NoSQL DB. Or it could just be a blob on disk. Whatever you like. Any database can handle this.

3. Many time based events.

I'm not seeing the word "data" here. Not a database issue.

4. A large number of player to player transactions - think the stock market.

Any database can handle this. That's what they do. If there's real money involved then you probably want to use a standard relational DB just so you can be sure about atomicity of transactions. But if you're just talking about item trading or whatever then that is not a big deal. It's more common just to do that in memory and use the DB as a backing store for the player saves. Save both players at the same time for better consistency.

Rule one of coding: constraints. What are your constraints?

Although brushing over this in a realistic way for most developers I'd say to KISS, just use mysql or some other kind of military grade database. It's often borderline impossible to predetermine how something will operate in a high stress environment in much the same way you can't figure out if your code will run fast without profiling it. You can make guesses, try to structure data and save/load methodology to make it efficient, but we're not gonna be able to throw out some magic number "Oh if you have 100k players you need XSQL and will need it in this format and saving this often." That doesn't exist.

On the off topic note: I don't get what people's obsession is with classifying it as MMORPGS(WoW, TSW, Eve, Wildstar, FFXIII, etc.) and "everything else" ORPG? I don't know. Usually when I talk to people about multiplayer games the term ORPG is useless and doesn't at all convey the wish for people to create a persistant game. I often see people go on a rant on here about how "THIS ISN'T AN MMORPG, AN MMO MUST BE MASSIVE!" to me that's utterly pedantic. A lot of the games we have long called MMORPGs like runescape or even ultima started off with very small amounts of people, often they only support small amounts of people and are just sharded to support insane numbers of people across insane numbers of servers. A lot of people seem to forget that many MMOS grow organically and don't start off supporting a million players. The MMO part is usually related to the type of game that is being created rather than the number of people on it.

On the off topic note: I don't get what people's obsession is with classifying it as MMORPGS(WoW, TSW, Eve, Wildstar, FFXIII, etc.) and "everything else" ORPG? I don't know. Usually when I talk to people about multiplayer games the term ORPG is useless and doesn't at all convey the wish for people to create a persistant game. I often see people go on a rant on here about how "THIS ISN'T AN MMORPG, AN MMO MUST BE MASSIVE!" to me that's utterly pedantic. A lot of the games we have long called MMORPGs like runescape or even ultima started off with very small amounts of people, often they only support small amounts of people and are just sharded to support insane numbers of people across insane numbers of servers. A lot of people seem to forget that many MMOS grow organically and don't start off supporting a million players. The MMO part is usually related to the type of game that is being created rather than the number of people on it.

FFXIII is a single-player game. I think you mean FFXIV.

It might be pedantic to you, that doesn't mean it's not a valid distinction for a lot of other people. Especially among developers, it's very common to use either different terminology, or terminology in a different manner than how consumers use it.

Terminology matters in communication, because it conveys a set of problems, solutions, and constraints.

Another important aspect is terminology changes over time. A concurrent user amount of x could be classified as massively multiplayer before, but not now. Just like a game that looked jaw-droppingly amazing 20 years ago would most likely not be perceived as all too amazing with the current visual standards.

Let's say I'm making a racing game, that is only viewed from the point of view of the driver. If I need to discuss it or find solutions to the technical side of things, do you think it's better for me to search for "First Person game" or "Racing game"? Stick a gun on the car and I could argue all day that I'm making a FPS game, but that doesn't help me one iota if I need to ask for help on car steering mechanics, friction and realistic engine sounds.

If what people/developers are actually asking questions about persistents worlds, then there is terminology for that, which you already used -- persistent.

MMO -- Massively Multiplayer Online -- conveys a problem set that includes a ridiculous amount of concurrent users, and all the follow-on effects of that. The solutions to those problems are, at this point, fairly well known, as is their cost: "very very very expensive".

A persistent online game, on the other hand, is a different beast altogether. Especially if it also comes with a estimate of concurrent users, and other important aspects.

I often see people go on a rant on here about how "THIS ISN'T AN MMORPG, AN MMO MUST BE MASSIVE!" to me that's utterly pedantic. A lot of the games we have long called MMORPGs like runescape or even ultima started off with very small amounts of people

I guess the issue is that it's important to know just what sort of scale we're talking about, because there are certain changes that can't really be made organically without massive upheaval. Ultima Online was certainly made with these large populations in mind (and pretty much invented the term 'sharding' for the way it approached the problem). Ironically, the (Western-made) game with probably the highest concurrency, World of Warcraft, is one of the most primitive in terms of having relatively few people per server.

Obviously there are some things that are common to "persistent online game" (RPG or otherwise), but there are things that vary based on the 'massiveness' (can I broadcast every update to everybody or do I need area of interest management? Can I service all players with one network adapter? Do I need to consider scaling horizontally? Can I keep all game data in memory?)

Apart from reality checks and practicality concerns, I think SwiftCoder is pointing right direction.

I think ideal setup would be

- A NoSQL database to store recent and relevant game data (if game data is tailored to benefit from document storage, therefore preferably no joins or aggregrated functions)

- A relational/NoSQL database to store logs, states and ancient data

- An in-memory solution such as Redis (Actually in theory Redis can be used mainDB although not a good idea, at least Redis makes your stack database-agnostic, theoretically you can use any DB behind)

But depending on how demanding game is and your player projection, you can even start with simple relational database. But in case like 10K concurrent player, it becomes a challenge out of scope of a beginner question that should be handled by professionals (who aren't including me :) )

I guess the issue is that it's important to know just what sort of scale we're talking about, because there are certain changes that can't really be made organically without massive upheaval.

That's my point though, some people are pushing "MMO" or "MMORPG" to be terms that mean literally something of the scale of the games I listed(and yes I meant FFXIV in my earlier post.) But most people, even a lot of developers I've met, will use MMO as a generic term for the kind of "large, usually persistent game." I don't think its a very descriptive term. Where do you draw the line? Facebook games? MUDs? Do we consider games like LOL to be MMOs because they have a large, complex background server architecture? Quite a lot of games have that. Heck a lot of mobile games would count as MMORPGs if we just go on that line of thinking, and that is an equally unhelpful basis for discussing the technical issues involved. A phone game won't have a lot of the issues that a WoW style game will have.

Ultima Online was certainly made with these large populations in mind (and pretty much invented the term 'sharding' for the way it approached the problem).

To be fair most MMOs start off generally small and end up being patched and improved over time to support the massive scale they usually fill.

Ironically, the (Western-made) game with probably the highest concurrency, World of Warcraft, is one of the most primitive in terms of having relatively few people per server.

WoW is another example of a game I think grew rather organically, I was there when it was released and the scale of people(and the back end servers likely) were much smaller, they redid a LOT of that game over the years, even before the expansions. But I mean.. putting it in perspective WoW was about as MMO as runescape was, they butchered the wc3 engine and turned it into an MMO.

Obviously there are some things that are common to "persistent online game" (RPG or otherwise), but there are things that vary based on the 'massiveness' (can I broadcast every update to everybody or do I need area of interest management? Can I service all players with one network adapter? Do I need to consider scaling horizontally? Can I keep all game data in memory?)

If anything all I've really thought writing these posts and reflecting on the issue is that games can have so many little pieces of each other that just blanketing them all under one technical term would never work. My main issue is how I often see us go off topic(like we are now) just to correct someone on "why you shouldn't be making an MMORPG instead of an ORPG" when instead the issue should be ascertaining what scale and specifics of a game they are talking about.
Although in this case I'd see the general response would be fine, that even making a small multiplayer online game can be very challenging.

Although in this case I'd see the general response would be fine, that even making a small multiplayer online game can be very challenging.

Even there you need to be more specific.

Is a multiplayer Facebook game played at the pace of actions-per-day (say, the long deceased Warbook) very challenging? Is a turn-based board/card game very challenging to make multiplayer?

I'd argue that most multiplayer games which you can implement using an off-the-shelf REST server are *not* very challenging. And off-the-shelf REST servers scale out to 10k concurrent users without even blinking.

But most people, even a lot of developers I've met, will use MMO as a generic term for the kind of "large, usually persistent game."

I've worked on MMOs, so I consider myself to have a slightly more useful definition of what it means than most people. That's not to say my definition is more correct, just more useful.

It's a useful term in that it gives you an idea of what people want to do. It's no worse than FPS, RPG, or any other term which may not fully capture all the requirements.

In this specific case, the OP said "If you are thinking of GW or WoW, stop. This project is not and never will be even approaching that scale." but goes on to say "10,000 concurrent" which is actually VERY large for a single server, if you're doing real-time simulation and broadcast rather than turn-based updates. (WoW, as I understand it, usually doesn't even approach 2/3rds of that in one world, and that world is both geographically partitioned and heavily instanced for dungeons.) So, as far as the "whoa, do you realise what you're getting in for?" aspect, I think some sort of "MMO-warning" is reasonable.