CIOs are overspending on the cloud — but still think it’s worth it

Major unanticipated increases in cloud spending, driven largely by increased use, aren’t spurring CIOs to rethink IT strategies beyond tactical shifts in how they attempt to rein in costs in the cloud.

According to a survey from Java platform provider Azul, 83% of CIOs say they’re currently spending more on the cloud than anticipated, with nearly half of those CIOs experiencing overages of 26% or more. Only 2% of CIOs report spending less on the cloud than they projected.

Cloud market observers point to large AI workloads driving up spending, but also to developers who aren’t clued-in to the cost of the cloud services they consume. And while many IT leaders have sharpened their cloud strategies to account for rising costs as AI comes online, the survey’s findings suggest there’s still a lot more work to do.

Repatriation, however, doesn’t seem to be a part of that work. Even as CIOs report major cloud spending increases, eight in 10 say their reliance on the cloud still saves their organizations money, placing the cloud market in a Jevons paradox, says Scott Sellers, president and CEO at Azul.

In a Jevons paradox, resources that tend to get cheaper over time can simultaneously experience higher levels of consumption, thus driving up spending. Moreover, even as CIOs spend more on cloud services than they have budgeted for, they seem to believe operating workloads on premises would be even more expensive, Sellers says.

The cloud also enables IT leaders to innovate and roll out new services, products, and features faster, he adds.

“In the old days, maybe you have a new feature that you rolled out, and it requires more compute infrastructure,” Seller says. “Then you called your local Dell sales representative, and three months later the servers arrive on your shipping dock. Many months after that, your IT guys got the servers installed.”

‘Not much we can do about it’

Pmfm.ai, provider of a platform for custom AI development, has seen its quarterly cloud spending go up by about 25% this year, says Aditya Saxena, the company’s founder. The main culprit, about 90% of the cost, is increased use, as the company’s customers create more research-oriented AI models that consume more tokens than traditional models.

Many customers also want dedicated cloud resources, such as their own server instances, he says.

Pmfm.ai has turned to special startup pricing offered by hyperscalers to control its cloud costs, but Saxena doesn’t expect a major impact. One cloud provider has predicted a 3% to 5% savings as it helps Pmfm.ai optimize its cloud infrastructure.

“Most of the services that we’ve spun up this quarter have been to mitigate unexpected traffic spikes and are preemptive in nature,” Saxena says. “It’s expected, and there’s not much we can do about it.”

Reining in cloud spend

Saxena’s experience is a common one for CIOs: Committed to the cloud, they must employ a variety of strategies to control cloud costs when spending climbs above expectations.

According to the Azul survey, the top strategies for doing so include optimizing workloads, leveraging cloud cost management tools, negotiating discounts, auditing cloud deployments, and adopting FinOps approaches.

Azul’s Sellers recommends an all-of-the-above approach. “There’s no single way to have an optimized cloud spend,” he says. “It takes everything, including really good observability and monitoring, so that you know exactly which application is consuming what resource, and then even finer levels of granularity about microservices within those applications.”

Organizations should also look at the types of cloud resources they consume, he advises. For example, sometimes a company will need cloud storage with super-low latency to run critical apps, but in other cases, it may be able to use high-latency cold storage.

Cloud cost Achilles’ heel: Developer education

CIOs should also embrace DevOps practices tied to cost reduction when consuming cloud resources, Sellers says. One pitfall that doesn’t get enough attention: Many organizations don’t educate developers on the cost of cloud services, despite the glut of developer services large cloud providers make trivial to call.

“I’ve lost track of how many services Amazon provides that developers can just use, and some of those can be quite expensive, but a developer doesn’t really know that,” Sellers says. “They’re like, ‘Instead of writing my own solution to this, I can just call this service that Amazon already provides, and boom, my job is done.’”

The disconnect between developers and financial factors in the cloud is a real problem that leads to increased cloud costs, adds Nick Durkin, field CTO at Harness, provider of an AI-driven software development platform. Without knowing the costs of accessing a cloud-based GPU or CPU, for example, a developer is like a home builder who doesn’t know the cost of wood or brick, Durkin says.

“If you’re not giving your smartest engineers access to the information about services that they can optimize on, how would you expect them to do it?” he says. “Then, finance comes back a month later with a beating stick.”

Businesses like banks that depend heavily on mobile apps are particularly vulnerable, Durkin says. But he recommends that IT leaders in any industry educate their developers about the cost of the cloud services they use — because the cloud’s key value proposition, scale, can hit your budget as well.

“Every time [developers] deploy something, it’s going to have cost in the cloud,” he says. “So if that artifact I build actually gets scaled out and delivered to multiple regions, multiple areas for ability, and if I overprovision it once, it gets overprovisioned 100 times.”