As chips and designs continue to push boundaries with ever higher Thermal Power Densities (TPDs), managing the dissipated heat becomes increasingly challenging.
Current air-cooled rack designs typically top out at around 10-20kW per rack. However, when you consider the Thermal Power Densities of CPUs and GPUs, it quickly becomes apparent that filling a rack with these heat-producing components becomes a limiting factor.
In a leaked Gigabyte roadmap, it has been revealed that most components are projected to double their power usage within the next 2-3 years. This power increase extends beyond CPUs to include networking components like higher bandwidth 400/800G switches, DPUs, FPGAs, and even optics.'
AMD's MI300X isn't listed here, but has increased power usage 34% from last gen, 560W to 750W.