AI in MMORPGs

MMOs face the basic problem that NPCs, once they encounter players, don't typically live very long. This leads to several problems when you want to have your NPCs learning intelligent behaviours. Furthermore, the next player they meet probably wont be that same as the last player they met and hence the strategy they try to employ will probably be inappropriate.

Personally, I think that MMOs do just find with having 90-95% of mobs being 'chaff' and the remainder being involved in encounters that last longer and require tactics. Take WoWarcraft as a good example. Most if not all mobs in the normal zones could be chaff. There to die quickly for the amusement and advancement of players. Mobs in instances could be made not tougher, but smarter; learning how to use the instance to their advantage and learning how to better adapt to the strategies that players come up with.

Most of this learning though should be conducted during offline analysis after a battle and a new strategy added to a 'book' of strategies that could be used depending on either the composition of the player forces or the rough strategy they employ.

Personally I think the fault doesn't lie with the AI developers for MMOs nor with the encounter designers, but rather that the designers simply don't have the tools to create 'intelligent' encounters and must rely on their experience, which necessarily limits the sorts of encounters that the player will face. Give the designers better tools and they'll do better work (usually).

Well, MMORPGs pose a very interesting challenge in itself. It is pretty much one consistent world all players play in. Having npcs that learn from player encounter over time poses many challenges in game balance in the long run.

Let's just assume that all low level npcs that are killed spawn from some sort of boot camp, which teaches them strategies. So, everytime you kill something, this "boot camp" learns something new to teach to the next generation. So, technically, as the game "ages" the NPCs will become more and more experienced. So, here comes the game balance issue. The purpose of low level NPCs is to help new players level up and get over the initial learning curve of the game, but now you have a generation of low level NPCs out there with some rather good tactics and harder to kill than they should be. This will completely throw off some of the game balance in the long run, as it becomes harder for people joining later to play the game or even learn it.

Even if we limit this learning to higher level or special event NPCs, this still throws things off somewhat as people who "came first" have the distinct advantage of going up against something "relatively easier" than say the 10,000 person to meet this monster.

So, in improving AI in an MMORPG, you may in fact be cutting out your target audience to just the people who started playing when the game was released or those who "have no life" and sit and play for long periods.

So, personally, I think more advanced AI hasn't been used in MMORPGs is just because they may cause game balance in the long run and may in some case discourage new people from joining.

The problem in this case should rather be, how do you create a predictable AI that is still hard to beat. You know all its moves, but talking about how to beat it is easier than the execution.

Personally, I feel the AI in MMORPGs should be there to make things hard enough to do on your own, but easy enough if you have a group of people who can work together in the right way, utilizing specific tactics. From my experience with FFXI, I've learned that everything sounds good and easy on paper until you really have to execute, that's when the difference between the player and noob is drawn.

The other huge constraint on MM ai is computation. There are likely a huge number of entities actively interacting with players at any given time. By increasing the complexity of any significant portion or entities you add a significant burden to the server's workload.

It seems to me its by necessity that the vast majority of entities can't become more complicated that simply reaction based and rail-driven.

Quote:

Original post by WeirdoFu Let's just assume that all low level npcs that are killed spawn from some sort of boot camp, which teaches them strategies. So, everytime you kill something, this "boot camp" learns something new to teach to the next generation. So, technically, as the game "ages" the NPCs will become more and more experienced. So, here comes the game balance issue.

This is something I've considered. Do you think it's conceivable that training the agents based on their observations/experiences may be counterproductive? Initially many MMO*s suffer from a particularly steep learning curve. Having agents that learn from what are essentially "dumb" players that are muddling through their first experiences of a game may result in less than optimal training.

Using one of my early MMO experiences as an example, the space-sim Jumpgate, the learning curve was incredibly steep and people often died to the very basic AI (not to mention splashed themselves against asteroids and stations thanks to the pseudo-Newtonian physics engine). Would training the agents based on this sort of behaviour result in them learning bad strategies, especially as the player's own skill improves quickly?

Andy

Quote:

Original post by Atavist The other huge constraint on MM ai is computation. There are likely a huge number of entities actively interacting with players at any given time. By increasing the complexity of any significant portion or entities you add a significant burden to the server's workload. It seems to me its by necessity that the vast majority of entities can't become more complicated that simply reaction based and rail-driven.

You said it before I could. AI is expensive and there are THOUSANDS of mobs running around that implement some sort of AI. It becomes a tradeoff between gameplay and cost.

Quote:

Original post by WeirdoFu So, everytime you kill something, this "boot camp" learns something new to teach to the next generation. So, technically, as the game "ages" the NPCs will become more and more experienced. So, here comes the game balance issue. The purpose of low level NPCs is to help new players level up and get over the initial learning curve of the game, but now you have a generation of low level NPCs out there with some rather good tactics and harder to kill than they should be. This will completely throw off some of the game balance in the long run, as it becomes harder for people joining later to play the game or even learn it.

If the AI is properly trained before the game is published, more experiance does not neccessarily mean that they actually get better. It just means that the same tactics will not work on them every time. As players change tactics, the NPCs will be forced to adapt as well but they would never be given more power and should never be able to find an optimal strategy.

Surely improved AI would call for vastly more computation power but in turn more clever enemies mean longer encounter and that we need less of them. Of course this may not compensate completely but should make it less of a sacrifice at least.

In addition, AI does not have to be restricted to enemies. Advanced dialogue systems and gestures could make neutral NPCs and players alike become more alive and beliveable; The interface could learn how you like to do things and make it easier for you to use the features you commonly use and cut out those you don´t; Running between places (corpseruns anyone?) could be automated with ligthweight but large-scale pathfinding algorithms etc etc.

That's why I'm actually fond of the idea of a dedicated bootcamp server for AI in MMORPGs and anything else. A server dedicated to evolving NPC behavior. When an NPC dies, they go back to bootcamp and gets evaluated as to whether they get retrained, sent back out again as is, or just deleted. This could work for MMORPGs and FPS bots alike, which could improve single player game replayability too if implemented correctly. The key is to have the proper performance metrics such that the bootcamp can actually learn "useful" things.

The hardest thing about designing MMORPGs, for me, is content. You are creating a virtual for thousands of players. This has to be a coherent world that obeys fixed physical rules, but yet be rich in content. Random content generation is nice and dandy until you hit coherency barriers. You don't want to have to go to an NPC and have them ask you to retrieve their dead son's corpse, and next day come back and have them ask you to save the same son from some dungeon. Just being able to have things make sense is very hard. You need to be able to believe that an NPC actually "lives" in that world, and that is the hard part.

Technically, everyone can make an MMORPG, just look at the situation in asia where there's like a constant 10 - 20 MMORPGs on the market at the same time. The problem is the content. Do you have a deep enough world that goes beyond the gloss of pretty graphics to keep players interested. I think just the focus itself on deep rich content over-rules the need for any form of complex AI.

In the end, I feel that to make a successful MMORPG, its not just pretty graphics or good AI even. Its the richness of this virtual world you created and how much content you actually have to lure players back again time after time. Is the experience immersive? Do I really feel that I am in a different world? Do things make sense? Is the story attractive? Finally, is it fun?

To ease the learning curve problem you could perhaps allow npcs access to some of the opponent data and change their attack tactics based on this. For example, in most rpgs you get the 'fodder' npcs that take a few blows to kill when you first start off but when you become harder they simply get annoying and pose little if any threat. If these npcs realsied you were a tougher opponent perhaps they could adopt group tactics joining into gangs before attacking so that the player is constantly challenged. Noobs can still improve their stats and get into the game and experiencd users don't grow tired of the creatures.

Group dynamics are one thing I meant to mention.
Having mobs not only attack in groups but give each other support. Trying to outflank, healing and buffing each other, causing diversions etc. I think this would enourmously improve the resusabilty of mobs and allow for some neat additional complexity.

Quote:

Original post by Froztwolf Group dynamics are one thing I meant to mention. Having mobs not only attack in groups but give each other support. Trying to outflank, healing and buffing each other, causing diversions etc. I think this would enourmously improve the resusabilty of mobs and allow for some neat additional complexity.

Come to think of it, the latest expansion for Final Fantasy XI added that where you had golbins that travelled in packs. They are nasty to take on because they will heal each other, buff each other up, debuff and enfeeble you and just basically try to swarm your mages if nearby. Taking them on by yourself is near suicide even if your level is much higher.... So, it turns out that everyone just end up leaving the packs alone because its just not worth the time to defeat them.

If you want to see a simple version of how NPCs are run in MMOs, go download the code for a MUD (developer.circlemud.org). These are very similar to how NPCs are handled in commerical MMOs, and you can expand it to allow your critters to do a crap load of interesting stuff aswell. Some things I've coded into this system:
team working (healing other npcs, or rescuing weaker ones)
summoning attackers
ranged aggression
ranged assistances (charging in, and assisting with ranged weapons)
racial assistance based on opponent strength
siegers (attacking cities)
over 30 types of aggressive flags

So much you can do with this kind of system (linked list), the only problem I have is finding time to add stuff :)

This might be an interesting resource:

http://www.cs.unimaas.nl/p.spronck/GameAIPage.htm

One of my profs created this learning AI (coined Online Adaptation) for Neverwinter Nights. It's not exactly an MMORPG, but it's close. The AI is equipped with various talents it can use in various ways to win a battle. A talent and the way it is employed (a tactic) is evaluated at the end of a battle, depending on the outcome (win/lose, HP left etc). This yields a table of weights indicating the usefulness of each tactic. Tactics with higher weights a more likely to be chosen during a subsequent battle.

This approach yields a completely self learning AI, which may be trained by pitting two team with the same AI against eachother. At first there was little distinction between teams with and without the learning AI, but after a few battles the learning AI started to gain a distinct advantage. By pre-setting the weights of certain tactis based on common sense, the learning AI can be made quite smart to begin with and because of the evaluation after each battle it was able to adopt to various attacks over time.

While I agree that "cannon-fodder" spawns are necessary for MMOs, they arent all that's necessary. Even with these, the AI in lots of single-player console games has better tactics than mobs in MMOs. At the very least, this should be upgraded.

The unreleased MMO "Dragon Empires" promised no static spawns. What does this mean in terms of AI...well, that if a group of orcs was killed in Dark River Glade, they wouldnt respawn there, but in turn move to SureStep Mountains.
This would have made the world semi-dynamic.

I want to take that process further and link that migration to objects and resources in the world that those beings need - e.g. water, food, power source (our game is post-apocalyptic). Not only would these creatures then be dynamic, they would be linked to the world/ecosystem, as well as creating gameplay by attempting to take over resources that players need as well. Instant dynamic conflict.

In another thread we were talking about NPC schedules. I brought up the idea of NPCs with dynamic goals based on constant needs assessment. This would take a lot of processing power, however you wouldnt use this setup for all NPCs just 100 say, out of all the ones on the whole server. Constantly fulfilling needs (from basic survival to actions based on philosophical beliefs (joining a resistance movement)) and reacting to the changing actions of players.

Its always been a /boggle to me that more MMOs dont have civilization sims running in the background. It just seems so elemental. EQ2 is starting to do this, but it's still not on a realistic time schedule.

I think its going to be crucial to come up with an engine (such an overused word - esp by me) for coordinating dynamic npc AI with player content to create much more responsive worlds.