Gen AI ROI falls short of expectations, but belief persists

AI, and gen AI in particular, are continuing to bombard the enterprise, but the gains to date haven’t been as big, nor come as quickly, as many business leaders hoped. That’s according to the fourth quarterly edition of Deloitte AI Institute’s State of Generative AI in the Enterprise report released on Tuesday.

“We’re seeing more of a focus on the pragmatic,” says Jim Rowan, applied AI leader and principal at Deloitte Consulting. He explains that enterprise leaders, from the C-suite down, wrestle with the tension between innovation and regulation, which has been a significant contributor to slowing down gen AI projects.

Deloitte surveyed 2,773 director- to C-suite-level respondents experienced with AI, piloting, or implementing gen AI for the Q4 survey between July and September 2024. The respondents were from 14 countries and seven industries: consumer; energy; resources and industrials; financial services; life sciences and healthcare; technology, media, and telecom; and government and public services.

While 78% of respondents said they expect to increase their overall AI spend in the next fiscal year, more than two-thirds said 30% or fewer of their gen AI experiments would be fully scaled in the next three to six months.

“[Gen AI] isn’t an easy thing to do,” Rowan says. “And it’s not just an AI thing. It’s all the areas around it that have to come into alignment: the data, security, governance, the controls, and the risk, legal, and compliance departments all working together with IT functions and business leaders.”

Key barriers to entry

Concern about regulatory compliance has proven a top inhibitor to organizations developing and deploying gen AI tools and applications. In the Q1 survey, 28% of respondents cited it as a barrier, but in this Q4 survey, that figure climbed 10 percentage points to 38%. Moreover, 69% of respondents reported that fully implementing a governance strategy to support their gen AI efforts would take more than a year to resolve.

That said, even as business leaders discover that implementing gen AI at scale is hard, the gains are coming. Rowan says organizations are largely shifting their strategic focus on gen AI from technology catch-up to competitive differentiation as they see positive results from their efforts. Deloitte found almost all organizations had measurable ROI for their gen AI efforts over the course of the year, with 20% of respondents generating an ROI of 31% or more.

Agentic AI also exploded onto the scene in the latter part of the year. It didn’t feature at all in the first few quarters of Deloitte’s survey, however, but in Q4, 26% of respondents said they’re exploring autonomous agent development to a large extent, and 42% reported they’re exploring it to some extent.

Gaining momentum

To date, the IT function has had the most success getting value from gen AI. Rowan says 28% of respondents say their most advanced gen AI initiative is in the IT function, with cybersecurity initiatives coming in second.

Rowan attributes success in IT to several factors, including healthy interest in the technology from IT teams, time savings from gen AI code generation capabilities, and automation of capturing requirements and testing. Drinking their own champagne, so to speak, is also helping IT functions develop their talent base and gain experience with tools. And that, Rowan says, points to the opportunity CIOs have to differentiate themselves strategically in the era of gen AI.

“CIOs in particular have the opportunity to be the change agent on this,” he says. “They’ve got to create the trust in the technology for the enterprise. That means working with HR — the talent functions — to think through how to build enterprise trust. CIOs have the opportunity to be leaders in that space.”

Just as important, he adds, CIOs must be responsible to manage the cost associated with scaling gen AI and find the right models and solutions for business challenges.

“They’re going to have to be very thoughtful about going down from large parameter to small parameter models to make sure they’re driving the right TCO,” he says. “They’ll also have to stay very close to the ecosystem through their relationships in the tech world, and understand what the start-up community is doing, as well as what independent software vendors and hyperscalers are doing. They’ve got to tightly stitch together that ecosystem and know these product roadmaps.”

Maintain realistic expectations

CIOs also need to focus on identifying and overcoming the hurdles to large-scale gen AI deployment. Expect to make mistakes, Rowan says, so ensure you have the ability to recover quickly and pivot.

Along the way, CIOs are likely going to have to do some expectation managing and level-setting for other C-suite executives. Deloitte’s survey found that while interest and excitement about gen AI remained relatively high overall, C-suite respondents, relative to others, had the highest levels of excitement and optimism with 21% reporting that gen AI was already transforming their organizations, compared with 8% of non-C-suite respondents.

C-suite respondents also showed less concern about barriers like trust, risk management, governance, and regulatory compliance, and had a rosier view of how quickly their organizations move and could address barriers to scaling and value creation. But 60% of non-C-suite respondents believe it’ll take 12 months or more to overcome scaling barriers.