I need help with my creature simulation game!

Utility functions are the typical route for this type of simulation.

Each potential action is scored and given a number. Weights are scaled based on the effect on motives.

For example if a creature is only 5% hungry they get a small score, but 80% hungry eating gets a large score. Choosing different foods each has its own utility score. Actions can have multiple utility values, an action might affect hunger, hygiene, and social, and they all sum up. Scores can be negative, tired creatures might get negative scores from actions that take work, positive from those that cause rest or recovery.

Buffs / debuffs can further alter them. "Time for work/school" in a sim game could add a huge multiplier to the tasks. For creatures, maybe "mating season" applies a multiplier as well.

The simulation might find 130 different actions available. Loop through the collection, sort by scores, prune it to the top x%, then pick any that remain. By limiting to the top percent of the highest rated, if one action is tremendously high it will be the only one they'll pick. If several options are all similarly useful to the highest score, then picking any of that subset randomly gives variability.

I'm actually working on this right now!

What you've described is a much more rudimentary system than what I've built. Instead of calling them "motives" like you, I call them "needs". One question you'll have to figure out: What happens if you have competing needs? Which one gets prioritized? You could hand key a huge needs prioritization list, but that's pretty labor intensive and it would also be context sensitive to the type of creature. For example, a rabbit would have a need to eat carrots, and once the rabbit eats a carrot, it's hunger need is satisfied. However, a zombie would have a need to eat brains / flesh, and even if the zombie does find a brain to eat, it's 'needs satisfaction' is broken, it constantly needs more brains to feed on. So, how would you rectify the difference in needs prioritization between a zombie and a rabbit? Would you need to create needs prioritization tables for each creature type in your game? That'd be an enormous amount of manual labor. Instead, what you'd want to do is let machine learning do its thing: let the needs prioritization be figured out dynamically as a function of the reward and cost ratios. (I'm being pretty hand wavey here).

You'll also want to bind actions and needs together, combined with object affordances. If your villager is hungry and needs to eat, they need to perform the "eat" action on an object with the "edible" affordance -- such as a sandwich.

You'll also need to do action planning with sub goals. Imagine your hungry villager sees a sandwich sitting on a table inside of a house, but to get to that sandwich, they'd have to navigate to a door, open the door, navigate to the table, eat the sandwich. Now you've got a planned action chain to achieve a goal. But what if your villager was out plowing a field before they got hungry? How do you get them to go back out to plow the field after they ate the sandwich? How do you get them to resume plowing the field after they go to sleep at night and its a new day? This suggests that you need multiple LOD's for your action planning: LOD0 is the immediate actions, LOD1 is the short term actions, LOD2 is long term action planning, etc. And each LOD has a terminus which aligns with an action in the LOD above it.

You'll also need a way to represent the game world to your agents and a way for them to understand the world they see around them and make sense of it to plan their actions.

Another interesting area to explore is the idea of incomplete information. Should your AI agent know that the merchant the next town over dropped the prices on iron ingots by 30% and factor that into their reward maximization calculations? No! So, how do your AI agents get incomplete information in a world which they can query omnisciently?

I would go into detail on how I solve all of these problems elegantly and use one common cognition framework for all creatures, but I've been legally advised to keep my mouth shut for the time being 🙁

Quest Machine works very similarly to everything described above. When a quest giver generates a quest, it:

Generates a world model - a list of facts about the world, such "there's a carrot in the field" and "there's a sleeping den below this tree." These things in the world are "smart objects," which have affordances and effects (e.g., eating the carrot satisfies the food need and removes the carrot from the world). In Quest Machine, affordances also have requirements. For example, a treasure chest may have a requirement that the interactor has a key in its inventory. If you want to read more about smart objects, there are lots of great papers about The Sims, such as this one: https://team.inria.fr/imagine/files/2014/10/sims-slides.pdf

Then it evaluates those facts against its own needs using utility functions and makes a weighted random choice of one fact. Utility functions typically return a priority value on a curve, such as low priority at 50% sleepiness but high priority at 90% sleepiness.

Then it uses a planning system to determine which precondition actions it needs to do in order to get the world model into the state to perform the affordance on the fact that it's chosen (e.g., obtain the key so that it's in the inventory before opening the chest). Quest Machine does this using a planning system system to the STRIPS planner Jeff Orkin used in F.E.A.R: https://www.gamedevs.org/uploads/three-states-plan-ai-of-fear.pdf - basically A* pathfinding, put on a graph of interconnected actions (e.g., "have key" leads to "open chest") instead of a graph of map locations. This results in a plan, which is a sequential list of actions to perform.

However, that might be overkill for your creature simulation game since you probably don't need complex plans. You can get really interesting behavior out of a list of needs (food, sleep, health, etc.) with associated utility functions, and smart objects in the game world. These smart objects need to advertise their affordances (e.g., addresses sleepiness) and typically only specify an animation & sound effect to play when the creature gets to the object. So when the creature needs to act, it identifies its most urgent need, chooses a smart object that satisfies it, pathfinds to the object, and performs the specified animation and sound effect. It's really simple, but with enough smart objects and animations it produces very convincing behavior.

A relatively easy enhancement is to continue evaluating needs on some frequency while pathfinding in case another need, such as fleeing danger or fighting an enemy, suddenly becomes significantly higher in priority.

BTW, when choosing a smart object, you can rank them based on how close they are, how well they address the need, etc. It's better to do that than just choosing one at random. However, as a first pass, just choose one at random or choose the closest one. Once you get that working, you can refine the way you select objects.

Interesting problem. I'd probably handle this by letting motives drive utility scores rather than having separate behavior priorities, so behaviors are selected dynamically based on the creature's current needs. I'd be happy to take a look at your codebase and brainstorm some approaches if you're interested.