Why your data layer is AI’s most critical climate technology

I have spent enough time building cloud and AI infrastructure to know one thing: Efficiency problems never show up where teams expect them. They tend to sit just beneath the surface, quietly shaping outcomes long before they appear in the metrics anyone is tracking. Most enterprise conversations still center on token pricing. That focus makes sense. The per-unit cost of using models has fallen quickly, and organizations want to understand whether AI can scale without pushing budgets out of bounds.

That question has become more complicated than it looks. At the same time, the nature of each interaction has changed. What used to resemble a straightforward exchange now involves retrieval and ongoing reasoning that unfolds across multiple steps. Even with lower unit costs, total consumption continues to rise because each request requires more underlying work. This creates a contradiction. Tokens are cheaper, yet overall compute demand keeps increasing. That compute demand has to go somewhere, and where it goes is a growing problem. 

The answer, increasingly, is the power grid. The International Energy Agency has warned that AI and data centers are becoming a major source of electricity demand, and Goldman Sachs Research has projected that data center power demand could rise 165% by 2030 compared with 2023 levels. Today, electricity consumption from data centers already amounts to roughly 415 terawatt hours, or about 1.5% of global electricity use, and has been growing at roughly 12% annually over the past five years. Much of the response has focused on models or hardware. Both matter, but they do not explain the full picture. In enterprise environments, a meaningful share of inefficiency begins before a model processes anything at all. It begins in the data layer.

The paradox: Cheaper tokens, more total compute

I have seen this pattern play out before. When unit costs drop, usage expands and everyone is surprised, every time. AI is following that same trajectory.

Lower token pricing has made it easier to integrate models into more workflows. Those workflows have also become more involved. Systems retrieve broader context and revisit outputs before returning a result. The additional activity outweighs the savings from lower pricing. In many cases, a system prompt alone can consume 2,000–5,000 tokens before a user even contributes input, which means the baseline cost of each interaction has already expanded. This is no longer just a financial consideration. Increased compute demand brings increased energy consumption, which is becoming a central constraint for enterprise AI.

Why agentic AI changes the energy equation

Average load: It rarely tells the full story. A platform can appear stable while quietly absorbing constant internal work that never surfaces to users. Agentic AI introduces that kind of persistent demand.

An agent continues operating in the background even after a user interaction ends. It checks for new inputs, revisits its internal state, then prepares the next action. That ongoing loop changes how infrastructure is consumed. Instead of handling activity in bursts, systems begin to carry a steady baseline load. Compute usage becomes continuous rather than intermittent.

This shift exposes inefficiencies that might otherwise go unnoticed. A slow query or a fragmented data source affects far more than a single request. The same issue repeats continuously, which increases both cost and energy consumption over time. This matters even more as inference workloads scale, with projections suggesting they could account for more than 40% of data center demand by 2030.

The overlooked source of energy growth: The data layer

In almost every architecture review I’ve sat through, attention gravitates toward model performance or infrastructure spend. The data layer gets treated as an afterthought. That assumption falls apart the moment you move into retrieval-heavy AI. 

Agents depend on access to reliable context. They pull information from operational systems as well as analytical platforms. When those sources are disconnected, each interaction requires additional effort to assemble a usable view.

That effort accumulates quickly. Data must first be located before it can be used. It often needs to be moved into another environment, then reshaped into a format the model can process. Only after that does it become useful for decision-making. Each step introduces overhead that consumes compute and energy. In practice, engineers can spend up to 40% of their time preparing data before it is even usable for downstream systems. The model remains the most visible part of the system, but the surrounding data work often determines how much energy the system ultimately uses.

Fragmentation as the root cause

Enterprise architectures evolve over time, and fragmentation usually reflects a series of reasonable decisions rather than a single mistake. The challenge appears when AI systems need to operate across all environments at once.

Fragmentation forces repeated work. Systems retrieve overlapping datasets because no single source is trusted as authoritative. Pipelines reprocess information that already exists elsewhere. Teams build parallel structures instead of relying on shared ones.

This pattern increases demand in ways that are easy to overlook. When retrieval becomes inconsistent, applications compensate by sending more context than necessary. Models then process larger inputs, which increases token usage without improving the quality of the outcome. What appears to be a model efficiency issue often traces back to data architecture. It also helps explain why as many as 60% of AI projects are abandoned before reaching production, often due to gaps in data readiness rather than model capability.

Why sovereign, unified architecture changes the math

I have seen organizations improve efficiency without changing models simply by reducing friction in how data is accessed. The good news is that the fix is closer than most teams assume. 

A more unified architecture shortens the path between a question and the data needed to answer it. When systems can access authoritative data directly, they avoid repeated transformations and unnecessary duplication. Retrieval becomes more precise, which allows models to operate with less excess input.

This does not require consolidating everything into a single system. Enterprises will continue to operate across multiple environments. The objective is to reduce unnecessary movement and make data easier to use wherever it resides. Sovereignty and control play an important role as well. Organizations operating across different environments need a clear understanding of where data resides and how it can be used. When governance is built into the architecture, systems spend less time reconciling access and more time producing results. Reducing friction at this layer has a direct effect on both cost and energy use.

What 2026 will expose

The next phase of enterprise AI will shift attention from access cost to operating cost. Early efforts focused on whether organizations could use advanced models. The next stage focuses on whether those systems can run continuously across real workflows without creating unsustainable demand.

That shift will make energy consumption more visible. It will also make inefficiencies harder to ignore. Some increase in demand reflects real value. Systems that improve decision-making or streamline operations will naturally require more compute. The more difficult question is whether additional consumption reflects useful work or avoidable overhead.

When systems repeatedly move and reprocess the same data, the increase in energy use does not correspond to better outcomes. It reflects architectural inefficiency. CIOs will need to examine that distinction more closely.

The takeaway for tech leaders

Before adding infrastructure, ask an honest question: Is this capacity supporting real growth, or is it just hiding inefficiencies that should have been fixed first? Start with the data layer.

Review how systems retrieve context. Look for duplication across environments. Identify where data is repeatedly transformed before it becomes usable. These patterns reveal whether the architecture is enabling efficient AI or creating unnecessary demand.

Enterprise systems will always involve complexity. The objective is to ensure that complexity does not translate into avoidable work. In the agentic era, the efficiency of AI systems is closely tied to how much work happens before the model produces an answer. Data architecture plays a central role in determining whether that work remains controlled or expands beyond what is necessary.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?