So I got to thinking about all the open source llm's out there now (especially the uncensored ones) and was wondering how practical it would be to incorporate that into a visual novel game.

My thoughts on this are a few-fold:

AI lack of 'knowledge' or 'reasoning' means that while they are amazing tools for assisting in ideas, conversations and content generation, they are terrible at 'taking the wheel' themselves. So I am strictly considering them in an 'assistive' capacity.

That means that a VN using AI should, in my opinion, have a predetermined story (issues of branching or linearity aside) and that there should be more or less pre-determined scenes and possible outcomes or 'exits' in scenes.

Where AI could be used is the in-scene comunication or generation. It could be used to generate story and character responses in scene and always be trying to 'steer' a scene to one of the 'exits'.

Obviously a lot would need to be firgured out with a system like this, but I feel like this overarching concept is not unreasonable.

This would of course need to run on a players cpu or gpu to be feasible as a game in my opinion, so the quality of your language models and techniques used are going to be limited.

This is a very open ended thought experiment on my part and I am wondering what the community at large has to say about it.

I had a similar idea which I might pursue for my game. I was going to use the LLM to generate the speech text of an AI personal assistant (similar to Cortana from Halo games). I was also planning to combine this with text-to-speech and speech-to-text models to allow the player to talk with the LLM verbally.

From what I've seen, some of the Llama 2 models (e.g. 7B, 13B) are feasible for this purpose. These have been quantized to 4 bits and converted to run on regular CPUs using 3.5GB and 7GB of memory, respectively, and can generate around 10 tokens per second on a single thread which is fast enough for human speech.

I agree with what you said - the LLM needs to be guided through the story. I was going to provide prompts to the LLM to describe locations the player is visiting and to provide context. Then the LLM can sort through that information and produce an interactive conversation based on the current context.

How well this works in practice remains to be seen. Combined with good text-to-speech models it could be used to replace much of the dialog in games (e.g. random conversations with NPCs).

@undefined Incorporating open-source language models into a visual novel game, especially for scene communication and content generation, is an intriguing idea. Utilizing AI as an assistive tool for dynamic in-scene dialogues and content steering within predetermined storylines can enhance player immersion and engagement. Balancing predetermined outcomes with AI-driven interactions could offer a unique and dynamic storytelling experience. Considerations for player agency and the AI's role in shaping the narrative will be key to a successful implementation. Exciting potential