CIOs sharpen cloud cost strategies — just as gen AI spikes loom

Cloud costs remain a key concern for IT leaders, who find themselves nearing a crossroads where expenditures for core workloads will need containment to free up spend for innovation.

To be sure, enterprise cloud budgets continue to increase, with IT decision-makers reporting that 31% of their overall technology budget will go toward cloud computing and two-thirds expecting their cloud budget to increase in the next 12 months, according to the Foundry Cloud Computing Study 2023.

Yet, controlling cloud costs remains the top challenge IT leaders face in making the most of their cloud strategies, with about one third — 35% — of respondents citing these expenses as the No. 1 barrier to moving forward in the cloud. Other top concerns are data privacy and security challenges (31%) and lack of cloud security and cloud expertise (24%).

“Cloud costs continue to be a top concern for CIOs,” says Dave McCarthy, analyst at IDC. He and other industry observers and IT leaders note, however, that an abundance of tools, managed services, and platforms are giving CIOs greater visibility into cloud costs and that alone is helping to lower the bills.

“Awareness of FinOps practices and the maturity of software that can automate cloud optimization activities have helped enterprises get a better understanding of key cost drivers,” McCarthy says, referring to the practice of blending finance and cloud operations to optimize cloud spend. “However, as these environments grow and become more complex, the challenges persist.”

And that is where many CIOs find themselves today: tackling cloud cost issues more skillfully just as disruptive forces such as generative AI are set to ensure those costs will exponentially escalate, CIOs predict.

The devil is in the details

One key skill CIOs are honing to lower costs is their ability to negotiate with cloud providers, said one CIO who declined to be named.

“People better understand the charges, and [they] better negotiate costs. After being in cloud and leveraging it better, we are able to manage compute and storage better ourselves,” said the CIO, who notes that vendors are not cutting costs on licenses or capacity but are offering more guidance and tools. “After some time, people have understood the storage needs better based on usage and preventing data extract fees.”

Thomas Phelps, CIO and SVP of corporate strategy at Laserfiche, says cloud contracts typically include several “gotchas” that IT leaders and procurement chiefs should be aware of, and he stresses the importance of studying terms of use before signing.

For instance, Phelps notes that discounts are often not applied to renewals and that there are no caps on price increases. Moreover, electing to downgrade service levels could nullify negotiated discounts, and security costs are often a separate line item that may require upgrading to an enterprise-level plan, which in turn could cost customers 20% to 30% more, he says.

CIOs may also fall into the trap of misunderstanding product mixes and the downside of auto-renewals, he adds.

“I often ask vendors to walk me through their product quote and explain what each product SKU or line item is, such as the cost for an application with the microservices and containerization,” Phelps says. “Most cloud agreements have an auto-renewal clause, which decreases leverage in negotiating prices upon renewal. I strike this language from all my contracts.”

Other CIOs manage their cloud costs by taking a hybrid approach: running some workloads on the public cloud where it makes sense and running heftier workloads on private clouds.

“It depends on what business model you’re in. If you’re in the business of data centers or perhaps have a heavy research and development arm, you might be able to do it more cheaply yourself,” says Ciena CIO Craig Williams, who admits this may be an outlier position. “We have moved our corporate environment to the cloud and have left our research and development organization alone to utilize internal data center resources since they’re large labs that require heavy network and compute loads. We’re constantly looking at the economics of both.”

Partnering for success

Not all CIOs are as experienced at cloud economics, and many are realizing it is best not to go it alone — at least in the preliminary stages of cloud deployment.

For example, Photogra, a 23-year-old image and photo provider for concession operators at amusement parks, cruises, and events, spent one year planning the migration of its data infrastructure from its New York data center to Microsoft Azure and other cloud services with the help of Aptum, a managed service provider.

Photogra President Michael Barlow says he wanted to move to the cloud to get out of the never-ending cycle of replacing hardware and servers, but he knew he would have to find an alternative cloud to store the images or run into oppressive Azure bills. He went with cloud provider Wasabi for those storage needs.

“We felt it was time to move to the cloud [in 2021] and provide this capability and flexibility, which would allow us to scale and grow without the need to reinvest in servers,” Barlow says, noting that Photogra’s databases and corporate data runs on Azure but that doing the same for the company’s extensive storage requirements may have made the cloud a non-starter.

Enterprises that try to migrate to the cloud on their own often run into cost and time overruns because of their inexperience, says Junaid Saiyed, CTO of data analytics firm Alation, adding that organizations that adopt self-hosted cloud solutions often do not know how to optimize the cloud’s computing, automation, and financial strategies.

“Opting for a managed cloud offering allows organizations to focus on delivering business value and driving adoption,” he says. “A self-hosted infrastructure is time-intensive, demands specialized expertise, and comes with significant expenses.”

Needing to hire IT talent to keep the train on the tracks — and the bills under control — is another budget issue many CIOs face.

“It’s the people cost — not what you’re actually paying Amazon,” says Thomas F. Famularo, managing director of life solutions at Verisk, who has seen an increase in enterprises using FinOps specialists. “You probably have three engineers that you’re paying more than $20,000 per month. People were complaining but they had to sharpen their pencils a little bit.”

Minding the meter

As IT leaders gain more experience with the cloud, they are also getting savvier about implementing effective strategies to cut operating expenses.

Zulfi Jeevanjee, EVP and CIO at Allstate, for example, says the insurance company has been “very diligent” about shutting off CPU usage when not in use. The CIO says he signed a contract with a cloud provider that gives him an out if he needs.

“If we start trending towards lines that we don’t like, there’s ways in the contract we negotiated and put in place to make sure our costs stay within the parameters that we set,” Jeevanjee says, noting that Allstate did have to commit to a certain level of services to negotiate this clause.

Ciena’s Williams, who is also seeing better deals on renewals, says that since he has a more predictable handle on his company’s own usage, he “can make more intelligent decisions/negotiations come contract renewal time.”

Gen AI: Cloud cost X factor

But even as IT leaders gain a steadier hand on the bottom line of cloud economics, emerging opportunities are complicated the equation.

ADP, a 75-year-old company with considerable experience running data centers, is a seasoned veteran in the cloud. Since it started moving to the cloud almost a decade ago, the company has implemented many tools, managed services, and governance procedures to cut costs on its hybrid multicloud architecture based on AWS.

“There’s a lot more transparency there,” says CIO Vipul Nagrath of the cloud cost landscape today.

Nagrath, who has served as ASP’s CIO for eight years, says Amazon’s Lambda serverless computing and Graviton servers have reduced ADP’s cloud costs considerably. Additionally, ADP’s management plan ensures all IT managers get daily reports and forecasts of cloud use to stay on top of the cloud spending.

But he and others — analysts and vendors alike — expect the substantial power and storage requirements of generative AI to significantly compound the costs to run those workloads.

“All of our [gen AI] experimentation started, which was something we couldn’t have predicted even six months ago, or a year ago, and so we have seen a spike [in costs] in those areas. But at least we know what’s causing it,” says Nagrath, who believes emerging applications such as AI will continue to make cloud cost management a perennial challenge.

In the meantime, CIOs efforts to contain spend may be combining to slow cloud sales growth somewhat, but vendors are not complaining. The Big 3 vendors — Amazon, Microsoft, and Google — are still selling tons of cloud software, just not at the speed of light anymore. IDC’s Public Cloud Services Tracker shows that worldwide spending on public cloud services will grow 22.2% year over year in 2023, which is down from the 27.6% growth experienced in 2022.

IDC expects global spending on public cloud services to total $656 billion in 2023, which is a “considerable increase” over the $537 billion spent in 2022. “I’m fairly certain all the cloud vendors have been reporting revenue growth this year, even if the growth isn’t as dramatic as in years past,” Rick Villars says.

And as gen AI use in production materializes, CIOs will have to deal with a whole new set of cost challenges. One thing for certain is that the costs will not come down. But for now, how high they will be is an issue for another day.

“There’s just not enough experience there to know what the ultimate costs for gen AI are,” says ADP’s Nagrath.