CIOs view cost management as possible AI value killer

CIOs feeling the pressure to deploy successful AI projects have a second concern: that they don’t have the money to pull it off.

Ninety percent of CIOs recently surveyed by Gartner say that managing AI costs is limiting their ability to get value from AI. In addition, if CIOs don’t fully understand the cost of scaling generative AI, they could miscalculate by 500% to 1,000%, says Hung LeHong, an analyst focused on executive leadership for digital business at Gartner. Depending on the AI project, a mistake of that magnitude could cost millions of dollars.

In many cases, using an LLM for simple AI tasks, such as transcribing and translating, can be expensive when cheaper tools are available, LeHong said during a recent webcast. “Some of the CIOs just don’t understand all of the cost elements that are there,” he adds. “Even if they do understand the cost, they don’t have the leverage to change it.”

Hidden costs and price hikes

Deploying AI takes a different approach than other technologies, adds Sumit Johar, CIO at finance software vendor BlackLine.

“The real issue is that cost of operationalizing gen AI isn’t always understood until you try to deploy it successfully,” he says. “It’s very easy to get quick success with a prototype, but there is hidden cost involved in making your data AI ready, training your AI models with corporate data, tuning it post deployment, putting the controls to limit abuse, biases, and hallucinations.”

BlackLine has deployed its own AI tools by focusing on projects that affect most employees and customers, Johar says. AI projects with broad impact show leadership and employees the power of AI, he adds.

But the deployment challenges associated with AI are ones many CIOs haven’t faced before, he adds. “We were used to writing rule-based systems that’ll always behave as you designed them,” he says. “Gen AI doesn’t work like that and requires several other investments to keep it in check while using it for what it was built.”

Beyond AI deployment challenges, software vendors are raising prices by 30% because of new AI features tacked on, Gartner says. While some software vendors are absorbing the added cost of AI for now, CIOs need to plan for the situation to change, LeHong says.

“Later on, those prices will go up,” he adds. “It’s not just the cost of the AI. It’s the cost of their applications that they’re already running in their business that’s actually a concern for them.”

A shift to small wins

The cost issue, combined with huge pressure from CEOs and boards to deploy and create value from AI, puts CIOs and other IT leaders in a difficult position, says Kevin Miller, CTO of industrial AI company IFS. In a recent IFS survey of IT decision-makers in industrial and related industries, 82% of respondents say they’re under significant pressure to adopt AI quickly, he notes.

“We all might have been a little guilty of running into the AI forest without necessarily knowing where we’re going or what that vision looks like,” he says. “The first thing that CIOs need to find is, where are the potential little wins with AI?”

Gartner’s prediction that CIOs can underestimate AI costs by 1,000% should be a wake-up call to CIOs to figure out how to measure and prioritize the AI projects that can provide value, Miller says.

The cost “just compounds exponentially,” he adds. “It really has the potential to go off the rails.”

In many cases, small wins that show quick value may be a better bet than huge, high-risk projects, Miller advises. He also recommends that CIOs interact with peer groups to learn about AI projects that have been successful.

“We can learn from others that have gone through this already,” he says. “How do I make this the most effective, meeting the demands of the board and the C-levels that I’m working with, by interacting with those peers to find out what worked and what didn’t and where they got the most value?”

Cost is certainly a concern when CIOs think about deploying gen AI, says Yuval Perlov, CTO at K2view, a data management vendor. In its own recent survey of senior AI deployment professionals, K2view found that cost was the top concern, followed by data security and privacy, as well as reliability of gen AI responses.

The problem, however, is as much a scale problem as it is a cost problem, Perlov says. In many cases, organizations launch ambitious AI projects that may be expensive to scale up when small and strategic initiatives may lead to quicker returns, he says.

“Some [CIOs] are playing around with technology, and they’re seeing cool things, and it’s not part of a strategy, and then they want to scale it up,” he says. “And often, it doesn’t scale up, either because they didn’t calculate it upfront, or they used an extremely expensive method to do it, like there are easier ways to achieve this, or cheaper ways.”

Unlike BlackLine’s Johar, who recommends deploying AI projects with a broad base of users, Perlov encourages IT leaders to think strategically and focus on use cases that have specific strategic implications for their organizations. In some cases, CIOs will want to sacrifice short-term ROI for long-term strategic advantages, and in other cases, CIOs will start small with easy-to-deploy projects that generate immediate wins to demonstrate to CEOs and board members.

Instead of creating a customer chatbot that answers 200 questions, start with one that answers the 10 most frequently asked questions, then offload the more uncommon questions to human agents, he suggests.

“Start small and deep and not wide and shallow, or do one thing or two things, do them very well to the point where you can roll it out and get value out of it,” Perlov says.

Doing so can help ensure costs are manageable and the solution will scale.