From Watts to Tokens

Every token that leaves a data center rides on three pipes: power coming in, heat going out, and data moving between tens of thousands of GPUs. The building can only do as much work as its narrowest pipe allows, and today that pipe is power.

Prefill is limited by compute, decode by memory bandwidth, and every doubling of context halves how many users a GPU can serve at once. How a model serves people decides how much memory and bandwidth the building has to hold.

Grid connections are slow, and by SemiAnalysis's count most of the queue isn't backed by real sites. Electricity is a rounding error next to the GPUs, so builders burn gas in inefficient turbines just to switch on a month earlier.

Power arrives at hundreds of kilovolts and leaves at about one volt at the chip. At every step down the voltage falls, the current climbs and resistive heat grows with its square. 800 V DC in the rack is the grid's old high-voltage trick moved indoors.

When the grid drops, capacitors catch the first milliseconds, batteries carry the next minutes and generators take the long haul: the earlier the runner, the faster it reacts and the less energy it holds. In the failures the course covers, the baton drops at the handoffs, like the one bus every power path shares.

NVLink inside the rack, InfiniBand or Ethernet between racks, fiber between campuses. A training run moves at the pace of its slowest card and most crowded link, so the network topology gets drawn together with the floor plan.

Air cooling lost on physics: moving 100 kW of heat takes roughly 3,500 times more air than water by volume. And whether a data center drinks water depends on which loop is open, since a sealed rack loop can still feed an evaporating tower.

Running one GPU for an hour costs roughly ten to forty cents of electricity and rents for two dollars and up. So the money in compute lives in the contracts: how long the price is locked, and who backstops the idle hours. Then Abilene ties the whole series together.