Data center
How it works
Servers and accelerators (GPUs/TPUs) are mounted in racks interconnected by a high-speed, low-latency network. Grid power passes through switchgear, UPS units (ride-through during outages) and backup generators before reaching the IT equipment. Heat is removed by cooling systems (air, liquid, or direct-to-chip), which become critical for AI clusters because of their power density. Energy efficiency is assessed with PUE; in efficient hyperscale facilities cooling is about 7% of energy use, versus often over 30% in enterprise sites. Availability and redundancy are classified by schemes such as the Uptime Institute Tier I–IV. Operationally, data centers split into enterprise, colocation, hyperscale and edge; per a 2024 LBNL report, hyperscale and colocation facilities held 74% of U.S. servers in 2023.
Problem solved
Modern computing — especially training and inference of large AI models — requires densely packed, reliable compute, continuous power and intensive heat removal that a single server or office server room cannot provide. A data center solves this by delivering a controlled environment with redundant power, cooling, high-speed networking and industrial-scale security.
Components
Compute units mounted in racks that perform the primary processing workloads.
Specialized chips for AI training and inference; the main driver of power density in AI data centers.
Disk arrays (SSD/HDD) and storage systems for applications and training datasets.
High-bandwidth, low-latency switches, routers and cabling connecting servers and linking the facility to the internet.
Switchgear, UPS units that sustain operation during outages, and backup generators ensuring continuity.
Air, liquid or direct-to-chip cooling systems; critical for AI clusters due to power density.
Ratio of total facility energy to energy delivered to IT equipment; ideal value 1.0. Standardized as ISO/IEC 30134-2:2016, developed by The Green Grid.
Implementation
Operators may omit some consumption (e.g. lighting) to report a lower PUE; the metric also ignores the site climate.
Grid constraints and local opposition delay or block projects; Data Center Watch recorded $64B of blocked or delayed projects (May 2024 – March 2025).
Intensive cooling can significantly strain local water resources, prompting environmental opposition.
Evolution
Early computers (e.g. ENIAC, 1945) occupied entire rooms — the first data centers.
The growth of the web drives dedicated server rooms and early colocation.
Launch of Amazon EC2 — the cloud model in provider-operated data centers (AWS, Azure, GCP).
Cloud operators build massive facilities; Open Compute Project (Facebook, 2011).
GPU clusters for training and inference drive record power demand; the IEA estimates ~415 TWh (~1.5% of global electricity) in 2024.
Compute bottleneck
In AI data centers the binding constraint is power delivery (grid interconnection, often over 3,000 MW for the largest facilities) and heat removal from densely packed accelerators, rather than chip count alone.
Hardware requirements
GPU clusters with tensor cores are today the primary workload of AI data centers (training and inference).
Cloud-operator data centers (e.g. Google) use TPUs for AI workloads.
CPU servers handle general-purpose workloads, cloud services and some inference.
As a facility, a data center can host any compute hardware regardless of accelerator type.