AI Engine
AMD hardware architecture for AI inference and signal processing
The AI Engine is a programmable hardware architecture developed by Xilinx and now AMD. It accelerates artificial intelligence inference and digital signal processing within Versal adaptive platforms and Ryzen AI processors. The design combines vector and scalar processors to handle parallel workloads efficiently.
- Originally developed by Xilinx before AMD acquisition in 2022
- Uses 7-way VLIW processor design for parallel execution
- Second generation supports bfloat16 and INT8 data types
- Integrated into Ryzen AI processors as XDNA architecture
- First Versal systems contained 400 AI engines per layer
What it is
The AI Engine is a computational core designed for artificial intelligence inference and high-performance computing. Originally created by Xilinx and now part of AMD, it serves as the intelligent layer in Versal adaptive compute acceleration platforms. The architecture streamlines linear algebra operations, such as matrix multiplication, and supports digital signal processing tasks. It integrates with field-programmable gate arrays (FPGAs), central processing units, and graphics processors to enable heterogeneous computation. Unlike general-purpose processors, this engine focuses on data parallelism and specific AI workloads, simplifying application development through dedicated software layers that manage memory and scheduling.
How it came about
Xilinx initially released the architecture to accelerate signal processing and applications benefiting from data parallelism. The first products, including the VCK190 and VCK5000, featured 400 AI engines connected via a Network on Chip. In 2022, AMD acquired Xilinx and integrated these engines into broader computing systems to address AI workload needs. Recognizing the complexity of the original platform, AMD worked with developers to improve toolsets for better programmability. In 2023, the company announced the second generation, AI Engine ML, which added support for deep learning data types like bfloat16. This evolution led to the XDNA architecture, found in Ryzen AI processors, with subsequent updates targeting large language model inference.
Inside the chip
Each AI Engine tile operates as a seven-way Very Long Instruction Word (VLIW) processor, allowing parallel execution of multiple operations per clock cycle. The design includes a 128-bit wide vector unit for Single Instruction Multiple Data (SIMD) tasks and a scalar unit for control logic. First-generation tiles contain 32 KB of local memory, expandable to 128 KB by using neighboring resources. The architecture uses static scheduling, which can require manual optimization to manage code size. Programming is primarily done in C++ to define kernel logic and connections between engines. The second generation, AIE-ML, doubles computing density per tile and supports optimized data formats like INT8 and bfloat16 for improved efficiency in machine learning tasks.
How it was made
The initial AI Engine layer combined 400 engines within Versal systems, utilizing a VC1902 for connectivity. These engines communicate through cascade, memory, and stream interfaces. The architecture relies on a Network on Chip for high-performance internal connectivity. Packaging involves integrating these compute tiles with memory tiles and shim tiles in the XDNA framework. The shim tile manages host memory interactions, while the memory tile handles data movement patterns to support the compute units.
Where it went
These engines are deployed in Versal adaptive compute acceleration platforms for industrial and embedded applications requiring high-performance signal processing. They are also integral to AMD Ryzen AI processors, such as the Phoenix and Strix series, where they function as Neural Processing Units (NPUs). In these personal computers, the engines handle energy-efficient AI inference and deep learning workloads. Researchers have also explored their use in space computing applications. The architecture supports diverse domains by combining with CPUs and GPUs, making it suitable for tasks ranging from digital signal processing to large language model inference on edge devices.
Versions and family
The first generation focused on DSP-heavy workloads and required explicit SIMD programming. The second generation, AI Engine ML (AIE-ML), offers three times more tera-operations per second per watt and enlarged memory for intermediate computations. XDNA 1 integrates AIE-ML with memory and shim tiles in Ryzen AI Phoenix PCs. XDNA 2, part of the Strix architecture, provides more units specifically tailored for massive machine learning workloads. AMD also refers to these blocks as Compute Tiles within the XDNA apparatus. The progression shows a shift from general signal processing to specialized AI inference support with enhanced data type compatibility.
Why it matters
The architecture has influenced the categorization of such hardware as Neural Processing Units, a term now used by various industry players. By integrating AI engines with standard processors, AMD has aimed to simplify the programming challenges associated with heterogeneous computing. The release of open-source tools like Riallto and the Ryzen AI SW toolchain reflects an effort to improve accessibility for developers. This approach contrasts with earlier complex systems, aiming to boost productivity in academic and industrial sectors. The design supports scalable edge-based machine learning, addressing the growing demand for efficient deep learning hardware in consumer and embedded electronics.
Questions people ask about AI Engine
What is the primary purpose of an AI Engine?
It serves as a computational core for artificial intelligence inference and digital signal processing. The architecture accelerates linear algebra operations and manages memory to simplify application development for AI workloads.
How does the AI Engine ML differ from the first generation?
The second generation offers nearly twice the computing density per tile and supports AI-specific data types like bfloat16. It delivers up to three times more performance per watt compared to the original DSP-focused design.
Which programming language is used for AI Engines?
C++ is the main language for defining kernel logic and connections between engines. However, research toolchains like IRON and ARIES allow users to specify tasks using Python, which is then translated into the required format.
What is the XDNA architecture?
XDNA is a hardware layer that combines Compute Tiles (AI engines), Memory Tiles, and Shim Tiles. It is integrated into AMD Ryzen AI processors to handle AI and signal processing workloads efficiently alongside CPUs and GPUs.
Good reading and sources
- AMD Completes Acquisition of XilinxAMD, Article
- AMD unwraps Ryzen AI 300 series ‘Strix Point’ processors — 50 TOPS of AI performance, Zen 5c density cores come to Ryzen 9 for the first timAnandTech, Article
- GAMA: High-Performance GEMM Acceleration on AMD Versal ML-Optimized AI EnginesarXiv, Article
The facts on this page come from the Wikipedia article "AI engine", written by its contributors and used under the CC BY-SA 4.0 licence, and from its Wikidata record (CC0). EveryChip rewrote the text in its own words, with AI assistance, and compares every page with the article to make sure no passage is copied. It may be reused under the same CC BY-SA 4.0 licence with a link back.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.
Like AI Engine
All AI acceleratorsMore from AMD
All of them
WorldAMD K9
Cancelled AMD microarchitecture intended to replace K8CPU · x86