News
DesignAdvancing chip design with OpenAI modelsDesignEMASS Launches ECS-Studio Development Suite for ECS-DoT Edge AIIndiaSmart Silicon: Building India’s Semiconductor Capabilities Through AIIndiaTata Electronics flags supply chain, talent gaps in ₹1.18 lakh crore semiconductor pushIndustryTSMC’s grip on advanced chipmaking will face competitionIndustryCadence joins European Chip Design Platform to support semiconductor startups and SMEsMaking chipsEurope plans to catch the next computer chip waveProcessorsAMD CES 2027 Invite Hints at Possible Ryzen Zen 6 "Olympic Ridge" RevealMaking chipsMarc Hijink on ASML’s Culture, Supply Chain, and GeopoliticsProcessorsIntel's post-Nova Lake CPUs take shape as Razor Lake desktop and laptop variants surfaceDesignAdvancing chip design with OpenAI modelsDesignEMASS Launches ECS-Studio Development Suite for ECS-DoT Edge AIIndiaSmart Silicon: Building India’s Semiconductor Capabilities Through AIIndiaTata Electronics flags supply chain, talent gaps in ₹1.18 lakh crore semiconductor pushIndustryTSMC’s grip on advanced chipmaking will face competitionIndustryCadence joins European Chip Design Platform to support semiconductor startups and SMEsMaking chipsEurope plans to catch the next computer chip waveProcessorsAMD CES 2027 Invite Hints at Possible Ryzen Zen 6 "Olympic Ridge" RevealMaking chipsMarc Hijink on ASML’s Culture, Supply Chain, and GeopoliticsProcessorsIntel's post-Nova Lake CPUs take shape as Razor Lake desktop and laptop variants surface
EveryChip

AI training and inference

Accelerators for training and running neural networks. 128 chips on EveryChip so far.

AI chips train neural networks and run them, a task called inference. Training a large language model takes thousands of accelerators running for weeks; inference runs every time someone uses the model. Nvidia's data centre GPUs have led since the deep-learning revival of 2012, while Google's TPU, AMD's Instinct, Amazon's Trainium and many start-up chips compete. Phones and laptops now carry small AI engines too.

Training and inference

Training adjusts billions of weights by passing data through the network and back, again and again, so it needs huge amounts of arithmetic and memory bandwidth and fast links between chips. Inference runs the finished model, often at lower precision, and is spread across data centres and devices. Many chip designs specialise in one or the other.

Why Nvidia led

In 2012 a neural network called AlexNet, trained on two Nvidia gaming GPUs, won the ImageNet image recognition contest by a wide margin. Nvidia's CUDA software, available since 2007, made its GPUs the default for researchers. Its A100 (2020), H100 (2022) and Blackwell (2024) chips became the main tools of the AI industry, and demand outran supply in 2023 and 2024.

Famous AI training and inference chips

Further reading

Photo credits on this page

The small pictures above come from Wikimedia Commons under free licences. Some are cropped to fit.

The weekly digest

Every Monday: the chip of the day, new pages on EveryChip, and the conferences and anniversaries of the week ahead.