NVIDIA ACE & Inworld AI Are Bringing Lifelike NPCs To Next-Gen Games, Contextually Aware With Distinct Personalities

NVIDIA is working with Inworld AI to bring lifelike NPCs in next-gen games while also upgrading its own ACE technology with advanced animation & speech features.

NVIDIA first showcased its ACE (Avatar Cloud Engine) technology at Computex 2023 with the aim of helping developers integrate unique voice, conversation, and motion AI models into the characters in-game. The technology was later upgraded further with NVIDIA's NeMo SteerLM tools which add an extra level of customizations to the attributes of each individual character. An interesting demo was showcased by the company which we talked about in detail here.

Now, NVIDIA is taking things to the next level with ACE as the technology now brings more realism and accessibility to AI-powered NPCs, avatars, and digital humans. The new ACE model allows devs to use various cloud APIs for automatic speech recognition (ASR), Text-To-Speech (TTS), Neural Machine Translation (NMT), and Audio2Face (A2F). All of these tools are available in the early access program and work great with popular rendering tools such as Unreal Engine 5 which is going to be a powerhouse for the development of next-gen AAA gaming experiences.

But NVIDIA isn't only working on its own technologies but also partnering with startups who are creating next-gen gaming characters through the use of Generative AI. One such startup is called Inworld AI. If you have been following us, then you would have heard about Inworld AI before, as we have talked about their tech on various occasions.

Not only Xbox partnered up with Inworld AI to make an AI-assisted game development toolkit that could enable dynamic stories, quests, and dialogue for future AAA titles; the tech has already been incorporated in existing games by modders with some examples including GTA V getting a whole new storyline with Dynamic AI NPCs and Skyrim getting additional "immersive" dialogue options using Inworld's AI generative AI model. There's also the upcoming MMO Avalon, which will be putting the technology to good use to allow players to easily generate their own content with contextually savvy NPCs that offer realistic interactions based on the player's approach.

NVIDIA's blog goes into further detail on Inworld's AI "Character Engine," which is expected to go open-source after securing major funding from the likes of Lightspeed Studios, Microsoft, Samsung, and more. The details reveal the three main layers of the Character Engine:

Character Brain Layer: orchestrates a character’s performance by syncing to its multiple personality machine learning models, such as for text-to-speech, automatic speech recognition, emotions, gestures and animations.

The layer also enables AI-based NPCs to learn and adapt, navigate relationships and perform motivated actions. For example, users can create triggers using the “Goals and Action” feature to program NPCs to behave in a certain way in response to a given player input.

Contextual Mesh Layer: allows developers to set parameters for content and safety mechanisms, custom knowledge and narrative controls. Game developers can use the “Relationships” feature to create emergent narratives, such that an ally can turn into an enemy or vice versa based on how players treat an NPC.

Real-Time AI Layer: The Real-Time AI layer ensures optimal performance and scalability for real-time experiences.

via NVIDIA

Of course, running the Character Engine requires powerful AI hardware, and for this purpose, InWorld AI is utilizing NVIDIA's A100 Ampere accelerators which offer the company the most efficient cost/performance value.

Technologies like NVIDIA ACE and Inworld AI's Character Engine truly push the boundaries of what is possible and what can be achieved through Generative AI. NVIDIA has many plans for AI in the future, including fully neural rendering approaches, and this is just a taste of what is yet to come. One of the biggest AAA titles making use of NVIDIA's ACE technology is expected to be S.T.A.L.K.E.R 2, which arrives next year and we definitely can't wait to see it in action using AI to drive the next generation of NPCs.