Dataflow and tensor designs
Systolic arrays, wafer-scale engines and other designs built for neural networks. 8 chips on EveryChip so far.
Dataflow and tensor designs are processors built around the arithmetic of neural networks rather than around a stream of general instructions. Many use a systolic array: a grid of multiply-add units through which numbers pulse in step, like blood through the body, each unit passing its result to its neighbour. Others spread thousands of small cores and memory across a huge chip, up to a whole wafer, so data rarely has to leave the silicon.
Systolic arrays
H. T. Kung and Charles Leiserson described systolic arrays in the late 1970s. Google's first TPU, used from 2015, had a 256 by 256 grid of 8-bit multiply-add units, 65,536 in all, working on matrix multiplications. Because each value is reused by many units as it passes through, the chip needs far fewer trips to memory than a CPU would for the same work.
Wafer-scale and spatial chips
Cerebras builds a single chip from almost a whole 300 mm wafer, with hundreds of thousands of cores and on-chip memory, to keep a large model's data close. Tesla's Dojo tiles many D1 chips into a training system. Graphcore, Groq, SambaNova and others laid out compute and memory in new ways. Nvidia's tensor cores, added in Volta, brought matrix units into the GPU itself.
Famous tensor design chips
Further reading
Nothing matches those filters yet. Take one off, or check back: new pages are written every day.