The Virtual Human Toolkit is now on GitHub

USC ICT has released the Virtual Human Toolkit on GitHub, making its platform for building embodied conversational agents publicly accessible. The toolkit is aimed at real-time virtual humans that can perceive users and respond with speech, facial behavior, and other nonverbal cues, without teams having to assemble speech, NLP, TTS, and sensing systems from scratch.

The stack is built for broad deployment: characters are authored in Reallusion Character Creator and can ship to the web, iPhone, Android, and Quest 3. A browser demo is available with no install, and free mobile apps are already live. That makes it unusually practical for teams experimenting with interactive characters, training sims, or research prototypes that need to be tested quickly across devices.

A notable technical detail is the vendor-agnostic AI layer. VHToolkit supports more than 20 cloud and local providers, including ChatGPT, Claude, ElevenLabs, Ollama, and OpenFace, all behind a common API with real-time switching. It can also run entirely on local hardware, which is important for studies or products where participant data must stay on-premises. Retrieval-augmented generation is supported as well, using plain-text custom knowledge.

USC ICT is also building VHToolkit Studio, a no-code browser tool for authoring virtual human studies with conditions, character backgrounds, and embedded Qualtrics surveys. Both projects are being shown at IVA 2026, and the toolkit is available under the USC-RL v3.0 license for non-commercial research and educational use. For developers, the main takeaway is that a...