Inferentia2
Second-generation AI inference accelerator from AWS
Inferentia2 is a machine learning inference chip designed by Amazon Web Services. It serves as the core component of Inf2 instances, offering high throughput for neural network workloads using specialised NeuronCore-v2 architecture.
- Delivers 380 INT8 TOPS per chip
- Provides 190 TFLOPS for FP16 and BF16
- Features 1 TB/sec DMA bandwidth
- Contains two NeuronCore-v2 cores
What it is
Inferentia2 is a purpose-built hardware accelerator for machine learning inference tasks. Developed by Amazon Web Services, it represents the second generation of their Inferentia line. The chip is designed to handle complex neural network operations efficiently within cloud data centres. It powers the Inf2 instance family, providing dedicated compute resources for deploying AI models at scale.
Inside the chip
The chip architecture relies on NeuronCore-v2 technology. Each physical unit contains two of these cores. The design supports various numerical formats, including INT8, FP16, BF16, cFP8, and TF32. It delivers significant computational throughput for these data types. The system includes direct memory access engines with high bandwidth to manage data movement. Instruction set extensions help control flow and shape handling for diverse model structures.
Where it went
This accelerator is deployed in AWS cloud infrastructure. It serves as the primary processing unit for Inf2 instances. Developers use these instances to run trained machine learning models. The hardware is optimised for inference, which is the process of using a model to make predictions on new data. This setup allows for efficient scaling of AI applications in production environments.
Questions people ask about Inferentia2
What is the primary function of Inferentia2?
It is a machine learning inference accelerator. The chip is built to execute trained neural networks efficiently, focusing on prediction tasks rather than model training.
How many cores are in each Inferentia2 chip?
Each chip contains two NeuronCore-v2 cores. These cores handle the computational workload for AI inference tasks within the AWS cloud infrastructure.
Which AWS instances use this chip?
Inferentia2 chips are found in Inf2 instances. Up to twelve of these chips can be included in a single instance to provide substantial inference capacity.
Good reading and sources
There is no English Wikipedia article on Inferentia2 yet, so the facts on this page come from the sources listed above: awsdocs-neuron.readthedocs-hosted.com/en/latest/general/arch/neuron-ha. Some are the subject's own pages, so its claims about itself are reported as its claims. EveryChip wrote the text in its own words, with AI assistance, and compares it with the sources to make sure no passage is copied.
It may contain mistakes, and figures change over time. Nothing here is investment or buying advice. If something is wrong, tell us. Last refreshed 2026-11-14.