What is a data center — and why does it matter?
A plain-language guide to the buildings behind AI: what goes in, what comes out, how AI facilities differ from traditional ones, and what companies look for when choosing where to build.
On this page
- What is a data center?
- How are AI data centers different?
- What happens inside an AI data center
- Using the right model for the task
- Inputs & outputs
- Efficiency (PUE, WUE, CUE)
- Site selection
- Key terms glossary
1. What is a data center?
A warehouse for computing — thousands of servers running 24/7 behind every video stream, banking app, and AI chatbot, kept alive by dedicated power, cooling, and fiber.
From small server rooms to 300+ MW campuses. Growth is driven largely by AI workloads.
How a data center connects to the grid
Electricity travels from a power plant across long-distance high-voltage lines, gets stepped down at a substation, then feeds the campus — often on a dedicated tap.
2. How are AI data centers different?
Power density
40–120 kW/rack
vs 5–15 kW traditional — up to 10× more power in the same space, and far more heat.
Cooling
Liquid-cooled
Air can’t keep up. Evaporative towers can consume millions of gallons a day.
Grid draw
50–100 MW
Per training cluster — the continuous load of a small city.
In one sentence: a traditional data center serves millions of small, quick requests; an AI data center runs fewer, far heavier workloads that demand extreme power density and advanced cooling.
Read more — why the difference matters
Traditional data centers run general workloads — web hosting, email, databases, video streaming — on standard CPUs drawing moderate power.
- Power density: AI racks packed with GPUs like NVIDIA’s H100 or B200 draw 40–120 kW per rack vs 5–15 kW for traditional servers. AI facilities need vastly more power per square foot and generate far more heat.
- Cooling: Standard air cooling can’t handle GPU heat loads, so AI facilities use liquid cooling (piping coolant to chips) or rear-door heat exchangers. Some rely on evaporative cooling towers that consume millions of gallons of water per day.
- Grid impact: When dozens of 50–100 MW clusters concentrate in one region (Northern Virginia, Central Texas), they strain the grid, drive up electricity rates, and require billions in new transmission infrastructure.
3. What actually happens inside an AI data center?
Two kinds of work with very different power profiles: training (building the model — one massive, months-long burn) and inference (using it — a smaller but endless drip, billions of requests a day).
| Attribute | Training | Inference |
|---|---|---|
| Duration | Weeks–months (one-time) | Forever (24/7) |
| GPU usage | Thousands in lockstep | Spread across clusters |
| Power profile | Steady, flat, 24/7 | Spiky, follows the clock |
| Total lifetime energy | ~20–30% | ~70–80% |
Rule of thumb: Training is a one-time, massive, steady burst to build the model. Inference is the endless drip of everyday use. Training gets the headlines; inference quietly dominates the long-run footprint.
Read more — training vs inference, in depth
Training — building the model
Engineers feed enormous datasets — much of the public internet, books, code — and the model adjusts billions of internal parameters until it can predict language well.
- Runs once per model, but for weeks or months without stopping.
- Thousands of GPUs in lockstep — a 50– 100+ MW cluster running flat-out, 24/7.
- A near-constant, city-sized electrical load that’s hard for a grid to absorb.
- A single frontier model can consume tens of gigawatt-hours — as much electricity as thousands of homes use in a year.
Inference — using the model
Your prompt goes to a data center, runs through the trained model, and a response comes back — usually in under a second.
- Runs constantly, forever — every chat message and search summary.
- Each request is small, but there are billions per day across all users.
- Load is spiky and follows the clock — easier to shift toward cleaner grid hours.
- Over a model’s lifetime, inference usually dwarfs training in total energy.
The full lifecycle, start to finish
One-time burst: Training & retraining are the biggest single energy draws
Never stops: Inference runs 24/7 for the life of the model
The cycle repeats: Each new model generation starts the process over
This is why AI facilities come in two flavors: training campuses built for massive, constant power, and inference campuses placed close to users for low latency. Some sites do both.
4. Using the right model for the task
A frontier model can use 10–100× more energy per response than a small one — and for most everyday tasks, the small model answers just as well. Sending every request to the largest model is like taking a semi-truck to pick up groceries.
The takeaway: the greenest AI request is often the one that never touches a giant model. Right-sizing — the right model, a short prompt, a cached answer when possible — cuts energy dramatically with no visible drop in quality.
How teams right-size in practice
- Model routing — a lightweight system sends easy questions to a small model and only escalates hard ones to a large model.
- Distillation — training a small, cheap model to mimic a big one for a specific task, keeping most of the quality at a fraction of the cost.
- Caching & retrieval — reusing past answers or looking facts up in a database instead of re-running the model from scratch.
- Shorter prompts & outputs — energy scales with tokens processed, so concise in-and-out means less compute.
Small “mini” models already handle the bulk of real traffic — classification, summarizing, autocomplete, simple Q&A — at a fraction of the energy. Large frontier models shine at hard reasoning and complex code, but are overkill for routine requests.
5. Inputs and outputs — what goes in, what comes out
What goes IN
Electricity — 50–300+ MW
Water — 1–5M gal/day
Land — 50–500+ acres
Fiber — Redundant paths
Hardware — Refreshed every 3–5 yrs
What comes OUT
Compute services — AI & cloud (the product)
Waste heat — Rarely recaptured in US
Noise — 50–70+ dB at property line
CO₂ emissions — Varies by grid mix
Jobs — 50–150 permanent
Tax revenue — Often reduced by abatements
6. How can data centers be more efficient?
The industry uses several strategies to reduce energy, water, and carbon footprint. Not all operators adopt all of these — and the gap between the best and worst performers is wide.
PUE comparison
Lower is better. Every 0.1 improvement saves ~7–10% total energy. Theoretical perfect PUE is 1.0 (impossible).
The six efficiency levers, explained
Power efficiency (PUE)
Total facility energy ÷ IT equipment energy. 1.0 = perfect (impossible); 1.1–1.2 = best-in-class (Google, Meta); industry average ≈ 1.55 (Uptime Institute, 2024). Every 0.1 reduction saves ~7–10% of total energy.
Liquid cooling
Direct-to-chip cooling removes heat far more efficiently than air — 30–50% less cooling energy, and increasingly required for AI GPU racks drawing 60+ kW.
Free cooling
Cold-climate facilities (Nordics, Pacific Northwest, Ireland) use outside air much of the year, drastically cutting water and chiller energy.
Renewable energy
Leading operators sign PPAs for wind and solar. The gold standard is 24/7 Carbon-Free Energy — matching consumption with clean energy hour-by-hour on the same grid, not just annually through credits.
Water efficiency (WUE)
Liters of water per kWh of IT energy. 0.0 = air-cooled; 0.2–0.5 = efficient evaporative; 1.0–2.0 = heavy use. Arid-region facilities are switching to closed-loop chillers that use zero water at the cost of more energy.
Compute efficiency
The cheapest watt is the one you never draw: raise server utilization (industry average is just 12–18%), optimize models (quantization and distillation cut inference energy 2–10×), right-size hardware, and schedule deferrable jobs into off-peak, high-renewable hours.
7. Where do companies build — and what do they look for?
Site selection is driven by a specific checklist of requirements. Understanding what companies prioritize explains why data centers cluster in certain regions — and why some communities are targeted more than others.
Site-selection factors by importance
Power is king — everything else follows. Without available grid capacity, no amount of tax incentives matters.
What’s often missing from this checklist: community input, cumulative impact on local water and power resources, noise standards, and long-term rate impacts on existing ratepayers. These are the gaps this tracker aims to make visible.
Could they build in your town? The full toolkit includes a Community Siting Evaluator — enter your town or address to see how it scores on these 8 factors, with auto-populated data from public sources.
8. Key terms glossary
| Term | Definition |
|---|---|
| PUE | Power Usage Effectiveness — ratio of total facility energy to IT energy. Lower is better. |
| WUE | Water Usage Effectiveness — liters of water per kWh of IT energy. Lower is better. |
| CFE | Carbon-Free Energy — electricity from zero-carbon sources (solar, wind, nuclear, hydro). |
| Hyperscaler | The largest cloud/AI companies that build their own data centers (Google, Microsoft, Amazon, Meta). |
| Colocation (colo) | A data center operator that leases space, power, and cooling to tenants. |
| Interconnection queue | The list of projects waiting for grid connection approval from the regional operator (e.g., PJM, ERCOT). |
| Moratorium | A temporary ban or pause on new data-center construction, usually enacted by local or state government. |
| PPA | Power Purchase Agreement — a long-term contract to buy electricity from a specific generator, often renewable. |
| Rack density | The amount of power drawn per server rack, measured in kW. AI racks are 40–120+ kW vs. 5–15 kW traditional. |
| GPU | Graphics Processing Unit — specialized chips (like NVIDIA H100/B200) that power AI training and inference. |
| Inference | Running a trained AI model to generate responses — what happens when you use ChatGPT, Gemini, etc. |
| Training | The initial process of building an AI model by processing massive datasets. Extremely energy-intensive. |
| Evaporative cooling | Cooling method that evaporates water to remove heat. Effective but water-intensive. |
| Liquid cooling | Piping coolant directly to server chips. More efficient for high-density AI workloads. |
| Marginal emissions | The CO₂ rate of the next power plant that would turn on to serve new load. The right signal for load-shifting. |
What to do with this
Now you know how these facilities work and what makes your community a target. Next steps:
Start here — the 5-step wizard The health risks, sourced How this reaches your bill
Sources: IEA Energy and AI (2025), Uptime Institute Global Survey (2024), EPRI Powering Intelligence (2025), Google Environmental Report (2024), US DOE Data Center Primer.