Why did NVIDIA introduce Avatar Cloud Engine (ACE) for Games in May with Convai technology? The company has a lot of partners and a world-class internal technology team. But Convai’s solution was the one that delivered the “Wow” factor.
We are incredibly proud that Jensen Huang, NVIDIA’s CEO, introduced his product with a shout-out to Convai in the Computex keynote a few months ago. It was a big stage, and the presentation generated a lot of buzz. Expectations changed.
NVIDIA demonstrated what Convai users already knew. AI-based NPCs can now take on capabilities similar to human players. It’s a revolution for gaming and several other applications.
Convai’s Generative AI for Game Developers
The rise of large language models (LLM), AI advances in avatar movement coordination, and the introduction of NPC contextual awareness are making games more realistic, more personalized, and more immersive. Convai is at the forefront of this transformation. NVIDIA’s ACE for Games announcement stated:
…NVIDIA collaborated with Convai, an NVIDIA Inception startup, to showcase how developers will soon be able to use NVIDIA ACE for Games to build NPCs. Convai, focused on developing cutting-edge conversational AI for virtual game worlds, integrated ACE modules into its end-to-end real-time avatar platform.
What’s Going On Here?
In the video, the player, Kai, begins conversing with the NPC Jin in his noodle shop. Kai’s interaction is very humanlike. He starts by asking how Jin is doing. Jin reacts appropriately, saying, “Unfortunately, not so good.” Another natural comment by Kai follows, “How come?” Jin then shares his worries about rising crime but does not move right to the critical information for the player. After another exchange, Jin provides information that adds context and moves the game’s story forward.
None of this is scripted. Kai could have engaged Jin in any type of conversation. Jin can engage in humanlike conversation because Convai’s generative AI character builder backs the NPC. That is why NVIDIA tapped Convai for its demonstration and why so many game makers have reached out and tested our technology.
Many games used to suggest questions you could ask NPCs because there was no option for flexibility. More recently, games adopted natural language understanding (NLU) AI models that allowed for some flexibility in player inputs. Still, all the responses were pre-scripted, leading to limited variety and a very unhuman-like experience.
Worse still, NLU-based solutions often misunderstand the player's comments and then resort to a “fallback” response such as, “I don’t understand,” or even something nonsensical. In our journey, we intimately understand this challenge. Our roots trace back to the world of developers, pioneers in crafting and refining conversational AI through the lens of Natural Language Understanding.
These misunderstandings and cascade of fallbacks can become very frustrating for players. They also undermine the game’s realism and immersive qualities. Game scenes with underskilled NPCs then devolve into puzzles where users attempt to discover what to say to move forward with the robotic-acting characters.
The Generative AI Transformation = Flexibility with Control
Everyone was amazed by the ability of ChatGPT to carry on extended, natural conversations. Convai recognized the opportunity presented by generative AI long before ChatGPT arrived. Our CEO, Purnendu Mukherjee, was on the AI applications team that built NVIDIA’s LLM NeMo and the Jarvis framework for conversational AI. He also supported game developers looking to add these technologies to create more engaging user experiences.
From that experience, I saw that game developers needed a layer of infrastructure to make AI-powered humanlike NPCs a plug-and-play feature. The LLM must go beyond simple generative AI-backed conversation and fuse the character’s backstory with what it talks about, its personality, its evolving narrative, and how it interacts with players. In addition, developers require scalability with minimized latency. And this all needs to be coordinated with NPC movement, expressions, and context recognition.
And game developers need to have some control. They cannot have a hallucinating LLM responding with nonsense, making out-of-context statements, or inappropriate comments. Convai introduced guardrails to monitor the LLM response and ensure the NPC stays in character.
Much more is required than simply adding an LLM to the mix. Performance, orchestration, AI safety, and developer ease of use are critical elements that take humanlike NPC conversation and representation beyond a demo to a viable production-ready asset.
The Transformation of Games and Virtual Worlds
NVIDIA’s May demo using Convai technology was a significant step forward. However, we have advanced the technology further in the past three months. Convai has more features and improved performance, and we’ve made it even easier for developers to add to their games.
We published a video discussing our product features and roadmap. The vision is about more than delivering an exciting conversation. Convai is creating NPCs that will be indistinguishable from human players, leading to more immersive, engaging, and fun gameplay.
Check out our Playground and create your character to see how Convai can elevate your game NPCs. Also, feel free to ask a question. At Convai, we are developers first. So, we understand your world.