AI training and inference
Accelerators for training and running neural networks. 129 chips on EveryChip so far.
AI chips train neural networks and run them, a task called inference. Training a large language model takes thousands of accelerators running for weeks; inference runs every time someone uses the model. Nvidia's data centre GPUs have led since the deep-learning revival of 2012, while Google's TPU, AMD's Instinct, Amazon's Trainium and many start-up chips compete. Phones and laptops now carry small AI engines too.
Training and inference
Training adjusts billions of weights by passing data through the network and back, again and again, so it needs huge amounts of arithmetic and memory bandwidth and fast links between chips. Inference runs the finished model, often at lower precision, and is spread across data centres and devices. Many chip designs specialise in one or the other.
Why Nvidia led
In 2012 a neural network called AlexNet, trained on two Nvidia gaming GPUs, won the ImageNet image recognition contest by a wide margin. Nvidia's CUDA software, available since 2007, made its GPUs the default for researchers. Its A100 (2020), H100 (2022) and Blackwell (2024) chips became the main tools of the AI industry, and demand outran supply in 2023 and 2024.
Famous AI training and inference chips
Further reading
🇺🇸 United StatesSycamore
Google's 53-qubit superconducting quantum processorQuantum · Other · 2019
🇺🇸 United StatesGP5
Analog co-processor for belief propagation and machine learning tasksAI accelerator · Other
🇺🇸 United StatesWillow
105-qubit superconducting quantum processor by Google Quantum AIQuantum · Other · 2024
🇺🇸 United StatesIBM Heron
156-qubit tunable-coupler quantum processor by IBMQuantum · Other · 2023
🇺🇸 United StatesTrueNorth
IBM neuromorphic chip mimicking brain structureAI accelerator · Other · 2014
WorldElectrochemical RAM
Non-volatile memory for analog deep learning accelerationMemory · Other
🇺🇸 United StatesField-programmable object array
Programmable logic device bridging FPGA and ASIC gapsFPGA · Other · 2006
Photo credits on this page
The small pictures above come from Wikimedia Commons under free licences. Some are cropped to fit.
- AMD Instinct: Мой Компьютер, CC BY 3.0, source
- Electrochemical RAM: Goldenvu, CC BY-SA 4.0, source
- Tesla Dojo: Steve Jurvetson, CC BY 2.0, source
- Sycamore: Google ( https://www.youtube.com/channel/UCK8sQmJBp8GCxrOtXWBpyEA ), CC BY 3.0, source
- Tensor Processing Unit: Norman P. Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan,, CC BY 4.0, source
- TrueNorth: DARPA SyNAPSE, Public domain, source
- Hopper: 极客湾Geekerwan, CC BY 3.0, source
- Blackwell: 极客湾Geekerwan, CC BY 3.0, source