4 recs for CIOs to optimize AI budgets and improve sustainability

In the client-server era, the penalty for inefficient programming, such as unoptimized database calls, was largely confined to application responsiveness. Today, in the AI era, code, architectural, and platform inefficiencies are no longer just a performance issue, they’re a financial and environmental liability. Left unchecked, poor code cascades into soaring token costs and spikes data center power consumption, directly undermining both cloud budgets and corporate sustainability goals.

AI’s impact on sustainability

By 2029, IDC projects that the number of actively deployed AI agents will exceed 1 billion worldwide, which is 40 times more than in 2025. And these agents will perform 217 billion actions per day.

To deliver on this demand, AI data centers are being built out at an unprecedented rate, with Gartner forecasting that global spending on data centers over the next three years will increase 31.7% to surpass $650 billion, driven primarily by hyperscaler cloud providers building out AI foundations, and optimizing servers for heavy AI workloads.

All this presents a significant strain on the energy grid as well as environmental sustainability, including:

• The power double-down: The International Energy Agency (IEA) projects that global data center electricity consumption will more than double from about 415 to 945 TWh by 2030, primarily fueled by energy-intensive accelerated computing for AI.
• The inference premium: AI workloads are vastly more demanding than standard web activities. A gen AI query consumes roughly 10 times the electricity of a conventional keyword search, or roughly 2.9 watt-hours as opposed to 0.3 watt-hours.
• Water consumption: Cooling these dense clusters is highly resource intensive. Global AI-related water demand is expected to reach 4.2 to 6.6 billion cubic meters by 2027.

The good news, however, is it’s not all out of the control of end user organizations and CIOs. Just as in the client-server era, through careful planning and execution, CIOs have the potential to significantly improve the performance, costs, and sustainability impacts of their AI application portfolio.

Here are four recommendations to maximize value as you look across your AI applications and infrastructure estate.

Revisit business objectives in light of AI

AI applications and platforms bring several new headaches for CIOs and CFOs in terms of FinOps. The variable nature of AI vendor billing due to variable monthly token costs is just one well-known example. To avoid unpleasant surprises, be sure to carefully review vendor contracts to decipher pricing models. Look for what’s included in seat-based license fees and what’s added as variable charges for agentic AI usage.

In addition, explore new metrics and KPIs such as intelligence per watt to help make sense of your return on AI. Just as miles per gallon helps us evaluate new car purchases, IPW can help to measure the computational efficiency of a system. It quantifies how much intelligence — typically measured in AI inferences, tokens processed, or model training iterations — a processor can deliver for every watt of electrical power it consumes.

According to Max Romanenko, chief engineering officer at relational database platform EDB, cost per query tells you almost nothing in an agentic world where autonomous systems are spinning up databases, pipelines, and queries around the clock. “The metric that matters is intelligence per watt, how much useful AI you get for every unit of energy you spend,” he says. “It isn’t just an environmental number, it’s also a performance indicator.”

With the measurements in place, you can then start to manage and optimize each layer in the AI stack from the infrastructure, or hyperscaler, layer to your own data and application layers.

It’s important to bear in mind that high token usage isn’t necessarily a bad thing. It depends on the net value delivered by each AI application and use case. Managing and optimizing the AI stack is important, but you’ll also want to measure the business value being delivered by each of these applications so you can measure your return.

Take a sovereign AI approach when evaluating hyperscalers

As you work with hyperscalers like Amazon, Google and Microsoft, it’s important to understand how they charge and how much, but also their environmental footprints. For example, by reading their sustainability reports, you can find out their annual water consumption across their global data centers and compare them with other providers.

In 2025, Amazon’s global data center operations used 0.12 liters of water per kilowatt-hour, which amounts to 2.5 billion gallons, or 5% of the annual water consumed by the metro Seattle area. The company has been able to operate more than seven times better than the industry average and have improved their water efficiency by 52% since 2021.

As demand for cloud computing and AI grows, water efficiency is another important metric for CIOs to monitor within hyperscaler ESG reports. While not at the same level of regulation as scope 2 and 3 greenhouse gas (GHG) emissions reporting, enterprises need to pay increasing attention to water use efficiency (WUE) with water scarcity becoming a growing risk for hyperscalers.

The key requisite at the infrastructure layer, though, is to ensure sovereign AI. This doesn’t mean you need to own everything, but you need control over your AI-driven operations when conditions change. With 71% of global executives stating that switching their primary AI vendor or model would be difficult if required today, it’s important to understand AI dependencies and be able to avoid vendor lock-in. 

Control efficiency at the data layer

The AI energy conversation has fixated on models and GPUs, but every agent, model, and inference call runs on the data layer beneath them, and that’s the one place a CIO can actually move the numbers.

“You can’t control consumption at the model layer,” says Romanenko. “Agents consume what they consume. But you can control efficiency at the data layer, and for most enterprises that’s the only real lever they have. Optimize search, retrieval, and vector indexing where the work actually happens and you cut compute, cost, and carbon at the same time. Ignore it, and it’s like running the heat with every window open.”

Ann Dunkin, distinguished professor of the practice at Georgia Tech, adds that CIOs who bring models in house and run them in their own infrastructure, or in the cloud infrastructure of their choosing, can have more control over the sustainability of inference, as well as of their costs and how their data is used.

Fine tune the application layer

When balancing a mix of commercial AI packages and custom-built code, costs can quickly spiral due to inefficient design and orchestration, redundant APIs, and unoptimized model routing.

With inference calls costing approximately 10 times that of conventional web queries, for custom AI applications, it’s important to design them to only use probabilistic code where necessary. Since many custom applications utilize a combination of both probabilistic and deterministic code, this is exactly where software developers need to make smart choices in their designs.

Other techniques to fine tune the application layer include semantic caching, intelligent model routing, and internal AI capability registries. “CIOs can implement intelligent routing solutions to select the most cost-efficient model for every prompt,” says Dunkin. “The most flexible routing solutions can drop into a user’s existing environment and orchestrate the actions of the company’s existing models.”

For CIOs looking to maximize the business value of every AI application in their portfolio, these new considerations, including new metrics, tools and approaches from the infrastructure layer all the way up to the application layer, should be an essential part of the equation.