NVLink
Nvidia's high-speed link between its GPUs, and between GPUs and CPUs, much faster than PCIe, used to join many GPUs into one system.
Also: NVSwitch, scale-up interconnect
Training large AI models needs GPUs that share data constantly. Nvidia introduced NVLink with its P100 GPU in 2016, giving far more bandwidth than PCIe, and added NVSwitch chips in 2018 so that every GPU in a server can talk to every other at full speed. With Blackwell, an NVLink domain can join 72 GPUs in a rack. Competitors and partners formed UALink in 2024 as an open alternative, while AMD uses its Infinity Fabric. Links of this kind are called scale-up networks, as distinct from Ethernet or InfiniBand between racks.