Memory bandwidth
How much data per second can move between a chip and its memory, measured in gigabytes or terabytes per second.
Also: bandwidth, GB/s, TB/s
Bandwidth is the width of the road to memory times the speed of traffic on it. A desktop with two channels of DDR5 manages under 100 GB/s; a graphics card with GDDR6X or GDDR7 reaches around 1 TB/s; an AI accelerator with six or eight HBM stacks reaches several TB/s, with Nvidia quoting 3.35 TB/s for the H100 SXM and 8 TB/s for the B200. Latency, how long a single request takes, is a separate measure. Many AI workloads are limited by bandwidth rather than by arithmetic.