News
DesignAdvancing chip design with OpenAI modelsDesignEMASS Launches ECS-Studio Development Suite for ECS-DoT Edge AIIndiaSmart Silicon: Building India’s Semiconductor Capabilities Through AIIndiaTata Electronics flags supply chain, talent gaps in ₹1.18 lakh crore semiconductor pushIndustryTSMC’s grip on advanced chipmaking will face competitionIndustryCadence joins European Chip Design Platform to support semiconductor startups and SMEsMaking chipsEurope plans to catch the next computer chip waveProcessorsAMD CES 2027 Invite Hints at Possible Ryzen Zen 6 "Olympic Ridge" RevealMaking chipsMarc Hijink on ASML’s Culture, Supply Chain, and GeopoliticsProcessorsIntel's post-Nova Lake CPUs take shape as Razor Lake desktop and laptop variants surfaceDesignAdvancing chip design with OpenAI modelsDesignEMASS Launches ECS-Studio Development Suite for ECS-DoT Edge AIIndiaSmart Silicon: Building India’s Semiconductor Capabilities Through AIIndiaTata Electronics flags supply chain, talent gaps in ₹1.18 lakh crore semiconductor pushIndustryTSMC’s grip on advanced chipmaking will face competitionIndustryCadence joins European Chip Design Platform to support semiconductor startups and SMEsMaking chipsEurope plans to catch the next computer chip waveProcessorsAMD CES 2027 Invite Hints at Possible Ryzen Zen 6 "Olympic Ridge" RevealMaking chipsMarc Hijink on ASML’s Culture, Supply Chain, and GeopoliticsProcessorsIntel's post-Nova Lake CPUs take shape as Razor Lake desktop and laptop variants surface
EveryChip

Dataflow and tensor designs

Systolic arrays, wafer-scale engines and other designs built for neural networks. 8 chips on EveryChip so far.

Dataflow and tensor designs are processors built around the arithmetic of neural networks rather than around a stream of general instructions. Many use a systolic array: a grid of multiply-add units through which numbers pulse in step, like blood through the body, each unit passing its result to its neighbour. Others spread thousands of small cores and memory across a huge chip, up to a whole wafer, so data rarely has to leave the silicon.

Systolic arrays

H. T. Kung and Charles Leiserson described systolic arrays in the late 1970s. Google's first TPU, used from 2015, had a 256 by 256 grid of 8-bit multiply-add units, 65,536 in all, working on matrix multiplications. Because each value is reused by many units as it passes through, the chip needs far fewer trips to memory than a CPU would for the same work.

Wafer-scale and spatial chips

Cerebras builds a single chip from almost a whole 300 mm wafer, with hundreds of thousands of cores and on-chip memory, to keep a large model's data close. Tesla's Dojo tiles many D1 chips into a training system. Graphcore, Groq, SambaNova and others laid out compute and memory in new ways. Nvidia's tensor cores, added in Volta, brought matrix units into the GPU itself.

Famous tensor design chips

Further reading

Photo credits on this page

The small pictures above come from Wikimedia Commons under free licences. Some are cropped to fit.

  • Tesla Dojo: Steve Jurvetson, CC BY 2.0, source
  • Tensor Processing Unit: Norman P. Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagarajan,, CC BY 4.0, source

The weekly digest

Every Monday: the chip of the day, new pages on EveryChip, and the conferences and anniversaries of the week ahead.