October DealsAmazon USOctober deal check: compare before you payAmazon US: current deals, useful picks and tech finds.Check DealsWindows FixRecommendedWindows errors stealing your time? Find the fix fastScan stability, cleanup and performance issues.Fix NowOctober DealsAmazon USDeal season is back - check today's better picksAmazon US: current deals, useful picks and tech finds.See Picks×
Skip to content
VGSources
Blog

NVIDIA ACE Plugins Bring Speech-Driven Facial Animation to Unreal Engine 5

NVIDIA ACE’s Unreal plugins combine AI-character tools, with Audio2Face-3D generating speech-driven facial animation for MetaHumans and compatible custom rigs.
Length7 min Posted Quest giverVGSources Team
Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

NVIDIA’s ACE plugins for Unreal Engine 5 connect projects to tools for speech recognition, language generation, voice synthesis and character animation. The component most directly tied to digital-human facial realism is Audio2Face-3D: it turns speech audio into facial animation data that can drive a MetaHuman or a suitably configured custom character. It can improve lip synchronization and expressive movement, but it does not by itself make a character photorealistic.

What NVIDIA released

NVIDIA ACE is a suite of Unreal Engine plugins, not a single all-purpose realism switch. Its components can be combined into an interactive character pipeline or used selectively, depending on what a project needs. NVIDIA describes the suite and its available downloads on the ACE for Games page; plugin setup and behavior are documented in the ACE Unreal Plugin documentation.

  • Audio2Face-3D analyzes speech audio and generates facial animation data.
  • Animation Stream receives streamed audio and animation data from NVIDIA’s animation service. Its animation data uses the same format as Audio2Face-3D, allowing projects to share a character-animation setup between the workflows.
  • ASR (automatic speech recognition) converts spoken input to text. NVIDIA documents an English-language model and sample Blueprint content.
  • GPT/LLM functionality sends prompts or requests to a language model and receives generated text; the documentation describes local execution for this functionality.
  • TTS (text-to-speech) supplies spoken dialogue using ready-to-use voices.

Speech recognition, language generation and voice synthesis can make a character more responsive in conversation. They are not facial-animation features in themselves; Audio2Face-3D is the suite’s most direct route from a spoken line to moving facial features.

How Audio2Face-3D affects facial realism

Audio2Face-3D takes audio and produces animation data that Unreal can apply to a character. In practical terms, that can make mouth movement track speech and give the face motion shaped by the audio and model output. NVIDIA provides a MetaHuman configuration sample, but the workflow is not described as MetaHuman-only.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
#1 Best Overall
Sale
GIGABYTE Radeon RX 9070 XT Gaming OC 16G Graphics Card, PCIe 5.0, 16GB GDDR6, GV-R9070XTGAMING OC-16GD Video Card
  • Powered by Radeon RX 9070 XT
  • WINDFORCE Cooling System
  • Hawk Fan
  • Server-grade Thermal Conductive Gel
  • RGB Lighting

Facial animation is only one part of a convincing digital human. The result still depends on the character’s facial topology and rig, blend-shape mapping, teeth and tongue, eye focus and blinking, head and body movement, materials, lighting, audio quality and runtime performance. ACE supplies animation and AI-character tools; it does not repair weak modeling, uncanny eyes, poor lighting or a character rig that cannot use the generated data.

How the speech-to-character pipeline fits together

A conversational character might use the components in this order:

  1. A user speaks, or the application provides prerecorded audio.
  2. ASR converts a user’s speech to text, if speech input is part of the experience.
  3. An LLM/GPT component generates a response, if the character needs generated dialogue.
  4. TTS turns the response into voice audio.
  5. Audio2Face-3D analyzes the audio and generates facial animation data.
  6. Unreal applies the animation to the character and renders the scene.

This is a possible integration, not a requirement to use every plugin. A cinematic using an existing voice recording may need only the facial-animation portion. A real-time conversational NPC may need speech recognition, response generation, voice synthesis and facial animation. The right design depends on which parts the application already provides and which it needs ACE to supply.

Supported Unreal versions and platforms

NVIDIA’s current installation documentation identifies Unreal Engine 5.5 and 5.6 on Linux and Win64 as tested and supported configurations. NVIDIA says other engine versions or platforms may build from source but are unsupported. The ACE download page lists version-specific packages, including Audio2Face-3D downloads for UE 5.4, 5.5 and 5.6; a download being listed for a version is not the same as the install documentation declaring that configuration supported.

What’s actually slowing this PC down?

Pick the symptom - the matching free tool is one click away.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.
Compatibility item What NVIDIA documents
Tested Unreal versions UE 5.5 and 5.6, according to the installation documentation.
Supported operating systems Linux and Win64, according to the installation documentation.
Other engine versions or platforms May build, but NVIDIA does not list them as supported in that documentation.
Version-specific downloads The ACE for Games page lists packages for particular engine versions; availability varies by plugin.

Check the install page and download matching your intended plugin and engine version before integrating it. NVIDIA’s changelog records evolving support, including UE 5.6 support and the removal of official UE 5.4 support in a later release. Its latest documentation branch is labeled 2.5, so avoid assuming that instructions or compatibility from an older plugin release still apply.

Rank #2
Sale
ASUS Dual GeForce RTX 3050 6GB GDDR6 OC Edition Gaming Graphics Card
  • NVIDIA Ampere Streaming Multiprocessors: The all-new Ampere SM brings 2X the FP32 throughput and improved power efficiency.
  • 2nd Generation RT Cores: Experience 2X the throughput of 1st gen RT Cores, plus concurrent RT and shading for a whole new level of ray-tracing performance.
  • 3rd Generation Tensor Cores: Get up to 2X the throughput with structural sparsity and advanced AI algorithms such as DLSS. These cores deliver a massive boost in game performance and all-new AI capabilities.
  • Axial-tech fan design features a smaller fan hub that facilitates longer blades and a barrier ring that increases downward air pressure.
  • OC Mode : 1500 MHz (Boost Clock)/Default Mode : 1470 MHz (Boost Clock)

Installation and character integration

NVIDIA’s documented starting point is the base plugin identified as NV_ACE_Reference. Installation requires a C++ Unreal development environment as well as a compatible engine and operating system. The overall setup then involves adding and configuring the plugin, choosing the relevant service or local component, and connecting its animation data to the character.

  1. Check compatibility: confirm that the plugin package, Unreal version and operating system match NVIDIA’s current installation requirements.
  2. Prepare the project: use an Unreal C++ development setup, following the linked Epic guidance from NVIDIA’s installation page where needed.
  3. Download and add the plugin: obtain the base NV_ACE_Reference plugin and integrate it into the project as directed by the version-specific NVIDIA instructions.
  4. Choose the workflow: configure Audio2Face-3D, Animation Stream, GPT or a combination, according to whether the project needs animation, streamed service output or conversational features.
  5. Connect the character: use the documented animation setup and, for a MetaHuman, the relevant sample and mappings. Custom characters may need additional rig and retargeting work.
  6. Validate with project audio: test representative speech and runtime performance before relying on the setup in production.

NVIDIA documents names including Apply ACE Animation, Get Default Blendshape Map, Face_AnimBP, mh_arkit_mapping_pose_A2F, RemoteA2F and LegacyA2F. Follow the labels and assets for the plugin version being installed rather than mixing steps across releases. The installation documentation marks the legacy Live Link interface as deprecated and says it is expected to be removed in a future release; new integrations should use the current interface.

One naming distinction matters when following setup material: NVIDIA Animgraph is a service that streams character-animation data, while Unreal AnimGraph is Epic’s animation-logic system inside Animation Blueprints. They are separate parts of the pipeline, not two names for the same feature.

Free tools Windows power users keep installed

One-click scans. No signup required.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Local processing, services and hardware considerations

The deployment model is mixed. NVIDIA describes ACE workflows that run locally as well as plugins that communicate with NVIDIA services. Local execution is documented for GPT functionality, and the changelog describes local Audio2Face-3D plugins using NVIDIA GPUs. Other workflows, including Animation Stream, connect to an NVIDIA animation service. Check the specific component’s requirements rather than treating the whole suite as offline or cloud-only.

Local processing can reduce reliance on network services, but it still consumes compute resources. Facial animation models, language models and Unreal rendering can compete for GPU capacity. NVIDIA’s general plugin documentation does not establish a universal GPU model or VRAM requirement for every configuration, so verify the requirements for the particular package and workload rather than selecting hardware from a blanket claim.

Rank #3
ASUS TUF Gaming GeForce RTX™ 5080 16GB GDDR7 OC Edition Graphics Card
  • Powered by the NVIDIA Blackwell architecture and DLSS 4. System Requirements: Minimum 850W PSU with 16-pin 12V-2x6 (12VHPWR) connector required. Verify before purchasing.
  • Military-grade components deliver rock-solid power and longer lifespan for ultimate durability. Compatibility: 348mm (13.7") length, 3.6 slots, 4.3 lbs. Confirm case clearance and slot spacing. GPU bracket included.
  • Protective PCB coating helps protect against short circuits caused by moisture, dust, or debris
  • 3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans
  • Phase-change GPU thermal pad helps ensure optimal thermal performance and longevity, outlasting traditional thermal paste for graphics cards under heavy loads
Independent reader supportYour contribution helps us test, update, and keep practical guides available for everyone.Support on Ko-Fi

MetaHumans, custom characters and Epic’s native tools

NVIDIA supplies a MetaHuman configuration sample, which gives MetaHuman users a documented starting point. A custom character is not automatically compatible just because it is a skeletal mesh: its facial rig, blend shapes, names and mappings must work with the animation data, and the project may need an Animation Blueprint integration, retargeting or facial-pose assets.

Epic’s MetaHuman plugins overview describes MetaHuman animation workflows involving audio, video and mobile devices. That is a natural comparison for teams focused on capture and Epic’s native MetaHuman pipeline. ACE is broader: it combines audio-driven facial animation with speech, language and voice components. The two approaches address overlapping but not identical needs.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

What changed across plugin releases

NVIDIA’s changelog lists changes relevant to integration choices: UE 5.6 support and the later removal of official UE 5.4 support; six local Audio2Face-3D plugins using NVIDIA GPUs; real-time input mode; streaming support for Sound Wave assets; runtime selection of an Audio2Face-3D provider; a C++ API for raw audio; and improvements to MetaHuman pose assets. Earlier releases added UE 5.5 support, local GPT execution, blend-shape multipliers and offsets, latent Blueprint nodes and animation-stream functionality. These changes make matching the plugin version to the project important, particularly when upgrading Unreal or moving between local and service-based providers.

Licensing and deployment terms

NVIDIA describes its listed on-device workflows as MIT-licensed on the ACE for Games page. That statement should not be read as a blanket license for every model, service, dependency or third-party component in a project. Before commercial distribution, check the applicable terms for the plugin source, each model, any NVIDIA service, voice assets and other providers, along with data-handling obligations and support arrangements.

When ACE is a practical fit

  • Consider it if the project targets a documented Unreal configuration, needs speech-driven facial animation, and can integrate a plugin-based workflow.
  • It is especially relevant when the team already uses MetaHumans or has a compatible facial rig, or wants a modular combination of speech, language, voice and animation tools.
  • Compare alternatives if the priority is performance capture rather than generated speech animation; if the project needs official support for consoles, macOS, mobile or non-NVIDIA hardware; if every component must work offline; or if the team needs a hosted conversational service with minimal engine-side setup.
  • Plan for technical-art work when using a custom character, because rig mapping and runtime validation remain project responsibilities.

For broader NVIDIA character workflows, Audio2Face also appears in NVIDIA’s ACE microservices and digital-human platform announcements. Those services are part of a wider ecosystem; they should not be confused with a guarantee that every Unreal plugin feature runs locally.

Quick Recap

SaleBestseller No. 1
GIGABYTE Radeon RX 9070 XT Gaming OC 16G Graphics Card, PCIe 5.0, 16GB GDDR6, GV-R9070XTGAMING OC-16GD Video Card
GIGABYTE Radeon RX 9070 XT Gaming OC 16G Graphics Card, PCIe 5.0, 16GB GDDR6, GV-R9070XTGAMING OC-16GD Video Card
Powered by Radeon RX 9070 XT; WINDFORCE Cooling System; Hawk Fan; Server-grade Thermal Conductive Gel
$842.14
SaleBestseller No. 2
ASUS Dual GeForce RTX 3050 6GB GDDR6 OC Edition Gaming Graphics Card
ASUS Dual GeForce RTX 3050 6GB GDDR6 OC Edition Gaming Graphics Card
OC Mode : 1500 MHz (Boost Clock)/Default Mode : 1470 MHz (Boost Clock); A stainless steel bracket is harder and more resistant to corrosion.
$257.22
Bestseller No. 3
ASUS TUF Gaming GeForce RTX™ 5080 16GB GDDR7 OC Edition Graphics Card
ASUS TUF Gaming GeForce RTX™ 5080 16GB GDDR7 OC Edition Graphics Card
3.6-slot design with massive fin array optimized for airflow from three Axial-tech fans; Auto-Extreme precision automated manufacturing helps ensure higher reliability
$1,814.90

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

Leave a Reply

Your email address will not be published. Required fields are marked *

Special offer. See more information about Outbyte and uninstall instructions. Please review EULA and Privacy policy.

More quests from Patch Notes

  1. How To Create Custom Stickers & Shoutouts In Monster Hunter WildsMonster Hunter WildsBlog20min
  2. How to Get XL Gogoat in Pokemon Legends Z-A (An Extra-Large Gogoat)Pokemon Legends Z-ABlog20min
  3. How to Get Rotting Lightbringer in Diablo 4Diablo 4Blog17min
Recommended PC Tool
Recommended PC Tool
Windows Errors? Fix Them Before They SpreadFree repair scan
Crashes, No Sound, or Screen Glitches?Free driver scan

Two free Windows tools

One Free Minute Could Fix That PC

Before you go - each of these free tools takes about a minute and tackles what quietly slows a Windows PC down.

Special offer. View Outbyte info, uninstall instructions, EULA, and Privacy Policy.