Why AI is forcing a rethink of data center cooling

For years, cooling has played a supporting role in data center design. Decisions have been driven primarily by compute, storage and networking requirements, while cooling systems quietly ensured everything stayed within safe operating limits. Most enterprise environments operated well within the capabilities of traditional air-cooling that was designed to sustain normal growth. This let organizations focus their attention on capacity, performance and cost of the compute.

That balance is now being disrupted.

Artificial intelligence is reshaping the thermal profile of modern data centers. As organizations roll out more powerful CPUs, GPUs and TPU’s to support AI workloads, heat generation is rising at a pace that many facilities were never built to handle. With AI in the picture, cooling is no longer simply an operational consideration. It is becoming a primary constraint and strategic differentiator on AI infrastructure growth.

The limits of air cooling are becoming clear

Although traditional air cooling continues to support many enterprise workloads effectively, its limitations are becoming increasingly evident as organizations deploy larger AI clusters with increasingly power-hungry CPUs and GPUs, generating heat at levels older data centers were never designed to accommodate.

Racks that once operated at 5–10kW are being replaced by AI systems drawing 60kW or more, with some high-end deployments exceeding 100kW per rack. At the component level, individual GPUs are drawing 700W–1,200W each, placing large amounts of heat into a very small space. This shift represents a step-change in thermal density that conventional air-cooling systems, typically effective only up to around 20–30kW per rack, struggle to handle efficiently.

At these levels, the challenge becomes structural. Air can only do so much. There’s a hard limit to how efficiently it can move heat, and simply increasing airflow or optimising ventilation isn’t enough to keep pace with the rate at which heat is being generated.

The consequence is a growing imbalance between compute capability and cooling capacity. Data centers are being forced to use more energy for cooling, while simultaneously managing higher thermal risk and operational complexity. In some cases, this also introduces performance constraints, as systems throttle workloads to remain within safe operating temperatures.

Because of this, more organizations are turning to liquid cooling — particularly direct-to-chip approaches.

How direct-to-chip cooling is addressing rising heat challenges

The main limitation of air cooling is its relative inefficiency at removing concentrated heat. Direct-to-chip cooling addresses this. Instead of relying on chilled air moving around the room, direct-to-chip systems put cooling exactly where it’s needed, by placing cold plates directly onto high-heat components such as CPUs and GPUs. Coolant flows through these plates, absorbing heat at the source before carrying it away for dissipation via a heat exchange system.

A direct-to-chip cooling system is made of several parts working together. Cold plates absorb heat straight from the chips, while a coolant distribution unit (CDU) manages the temperature, pressure and flow of the liquid. The coolant moves through pipes connected to each rack, carrying heat away from the servers and into the facility’s wider cooling system while sensors monitor temperatures, flow rates and leak detection.

Liquids transfer heat far more efficiently than air, so direct-to-chip cooling allows significantly greater thermal loads to be managed with lower energy overheads. Because heat is removed more directly and effectively at the source, data centers require less power for fans, airflow and chiller operation, reducing overall energy consumption.

In most cases, only the components that generate the most heat are liquid-cooled. The rest of the system continues to rely on familiar air-cooling approaches. That mix is a big part of the appeal. A hybrid cooling approach allows organizations to improve cooling performance where it matters most, without having to redesign their entire environment.

Direct-to-chip isn’t one-size-fits-all

While direct-to-chip is talked about as a single approach, there are actually a few different ways to implement it. Most organizations use single-phase liquid cooling, where the coolant stays in liquid form throughout the process. It’s simple, easy to manage and fits well with existing operational models, which makes it a natural starting point.

But there is also a growing shift towards warm-water cooling. Because water is so effective at absorbing heat, systems don’t need to run at the same low temperatures as traditional air-cooled environments. This can reduce the need for energy-intensive chilling and improve overall efficiency.

In some setups, direct-to-chip cooling is paired with rear-door heat exchangers. These capture any remaining heat as air leaves the rack, helping to push densities even higher without overloading the system.

Ultimately, there isn’t a single “correct” way to approach cooling. The best method depends on the workloads being supported, the constraints of the facility and the organization’s longer-term plans. What’s clear, however, is that flexibility is becoming increasingly important as cooling requirements continue to evolve.

Direct-to-chip vs immersion cooling

As liquid cooling gains traction, direct-to-chip is often compared with immersion cooling. While both approaches address the same fundamental problem — removing significantly higher levels of heat – they do so in very different ways, with different implications for how data centers are designed and operated.

Immersion cooling takes a more radical route by fully submerging servers in dielectric fluid — a liquid that does not conduct electricity or conducts it extremely poorly. From a cooling perspective, it is highly effective and can handle extremely dense, high compute environments. But it comes with trade-offs.

Immersion cooling requires a rethink of how data centers operate. It also demands significant infrastructure shifts, which can make adoption challenging for organizations with established data center models.

Direct-to-chip cooling, on the other hand, offers a more gradual step forward. In general, servers keep their familiar design, and day-to-day maintenance doesn’t change dramatically. Teams can continue working in ways they already understand, making it a more practical step for many organizations.

This practicality makes all the difference. For most organizations, the decision isn’t just about which solution performs best in theory, it’s about what can be deployed, managed and scaled within the realities of existing operations. In that sense, direct-to-chip strikes a balance between performance gains and operational continuity, making it a more accessible starting point for many data centers navigating the shift to higher-density workloads.

Cooling as a competitive advantage

Cooling is no longer simply an operational concern. It is becoming a defining factor in how data centers scale, how efficiently they run and how reliably they perform. As AI workloads push infrastructure to new limits, the ability to manage heat effectively will directly influence how far and how fast organizations can grow.

That shift is also changing who owns the conversation. Decisions that once sat with facilities teams are now firmly on the agenda for CIOs, CTOs and infrastructure leaders. Thermal design, energy efficiency and cooling architecture are no longer niche considerations, they are central to cost control, sustainability targets and overall competitiveness.

At the same time, there is no one correct solution. Air cooling will continue to support many workloads, while immersion cooling will remain relevant for specialised, high-density use cases. Direct-to-chip cooling sits between the two, offering a practical way to handle increasing thermal demands without disrupting established operating models.

For organizations planning the next phase of their infrastructure, cooling can no longer be treated as an afterthought. It needs to be considered alongside compute, storage and networking from the outset.