How Ten Thousand GPUs Talk
NVLink inside the rack, InfiniBand or Ethernet between racks, fiber between campuses. A training run moves at the pace of its slowest card and most crowded link, so the network topology gets drawn together with the floor plan.