The NVIDIA Avatar Cloud Engine (ACE) is changing the future of non-playable characters (NPCs) in video games through the injection of intelligence. Generative AI technology has made this possible revolutionizing how gamers interact with a game world in real time. Imagine a virtual space where a player communicates with NPCs in an “intelligent, unscripted, and dynamic way.”

These NPCs possess persistent personalities that evolve while forming accurate facial expressions based on the players’ native language. 

According to an NVIDIA press release, NVIDIA ACE for Games is a custom AI model foundry service that aims to “transform games by bringing intelligence to non-playable characters (NPCs) through AI-powered natural language interactions.”

Third-party developers of middleware, tools, and video games can use the ACE technology to build customized conversation models and AI toolsets for further integration into their software programs. 

“Generative AI has the potential to revolutionize the interactivity players can have with game characters and dramatically increase immersion in games,” said John Spitzer, vice president of developer and performance technology at NVIDIA. “Building on our expertise in AI and decades of experience working with game developers, NVIDIA is spearheading the use of generative AI in games.”

“This is just going to end up being a repeat of procedural generation all over again.”, Reddit user, grady_vuckovic commented on the r/pcgaming subreddit. 

While some remain unamused by the unveil, others are excited about the future of video games.

“Holy (expletive), this is so good. After witnessing this, I can’t wait for it to become available;” said NNNcounter on r/pcgaming subreddit. 

Tools such as NVIDIA ACE have fostered new developments to improve their ever-expanding AI foundation models. NVIDIA NeMo provides developers with the toolset required to tune in-game character models. NeMo enables character backstories and personality types that fit the developer-created game world. Additionally, developers are capable of aligning player interactions according to the context of a scene with a set of programmable rules for NPCs. 

On the other hand, NVIDIA Riva provides automatic speech recognition and text-to-speech to deploy in supported technologies such as NVIDIA NeMo to produce live speech conversations. 

And, NVIDIA Omniverse Audio2Face works alongside the other two foundation models generating realistic facial expressions for in-game characters during a conversation gathered locally from a single audio source. 

Audio2Face features Omniverse connectors for Unreal Engine 5 so developers can utilize the toolset to apply facial animations directly to their MetaHuman characters. 

To demonstrate how NVIDIA ACE will begin to transform the video game industry, partner, Convai and NVIDIA prepared a Kairos Demo to show the current capabilities of the toolset. 

The Kairos Demo leveraged the technologies of Riva, Audio2Face, and NeMo to deliver and power the output in conversation. The seamless integration of the three modules wouldn’t have been possible without the relentlessly effortful collaboration of Convai. 

Jin and his Ramen shop, the demo scene created by the NVIDIA Lightspeed Studios rendered fully in Unreal Engine 5. This was possible using the power of NVIDIA RTX Direct Illumination for the lighting, shadows, and ray tracing. DLSS was also deployed to maximize the frame rate and image quality. 

The NVIDIA ACE demonstration being shown in real-time could be a testament to the potential of all the newly emerging technologies working in tandem to achieve realistic conversation in player-NPC interaction. 

And the same was true for the GDC 2024 tech demo, Covert Protocol from NVIDIA and Inworld. In the Covert Protocol tech demo, you play as a private detective who is tasked with various objectives as she interacts with the uncertain game world. The outcomes of conversations alter the gameplay narrative, presenting a new variation each time for players to experience. 

The implications of what this means for the replayability of video games are yet to be determined, but it calls for further variation and value in multiple playthroughs. Single-player-focused video games may begin to evolve and be at the forefront of the industry. 

The AI-driven levels impact real-time decision-making and player interactivity with the game world ensuring that no playthrough is the same remaining distinct from one another. To support developers, NVIDIA will be releasing the source code of the Covert Protocol tech demo so they can start implementing the technology into their titles. 

“The combination of NVIDIA ACE microservices and the Inworld Engine enables developers to create digital characters that can drive dynamic narratives, opening new possibilities for how gamers can decipher, deduce, and play.” said Kylan Gibbs, CEO of Inworld AI.

However, the developers of S.T.A.L.K.E.R. 2 Heart of Chernobyl have utilized AI foundational models such as Audio2Face in their title. Another title, Fort Solis has as well. The good and bad of the emergence of AI and generative tools like these in games are unknown. Despite the recent reports of AI adoption in various indie studios, it’s unclear whether these tools will harm or be beneficial to the future of video games. 

If you’d like to learn more about generative AI technologies, the NVIDIA Developer blog has the latest information. For more on Inworld’s collaboration with NVIDIA, click here.