CIOs’ AI confidence yet to match results

A huge majority of IT leaders believe they will meet or exceed AI expectations, but that confidence is not yet supported by key metrics for success. In fact, most have a long road ahead.

According to a new survey from AIOps observability provider Riverbed, 88% of technical specialists and business and IT leaders believe their organizations will make good on their AI expectations, despite only 12% currently having AI in enterprise-wide production. Moreover, just one in 10 AI projects have been fully deployed, respondents say, suggesting that enthusiasm is significantly outpacing the ability to deliver.

The Riverbed survey echoes other studies, including a recent report from MIT saying 95% of gen AI pilot projects fail.

In addition, while companies represented have doubled their AI investments in the past year, just 36% of respondents say their organizations are ready to fully use AI.

The survey also shows that executive-level leaders are more optimistic about AI than IT staff, says Jim Gargan, CMO at Riverbed. One major impediment appears to be data quality and consistency.

Survey respondents expressed doubts about whether the quality of their data was sufficient for AI, although many respondents see improvements. Still, only a third of respondents rated their data as excellent for relevance and suitability and for consistency and standardization, while less than half rated their data as excellent for quality and completeness, for accuracy and integrity, and for accessibility and usability.

“The promise is high, but the progress is a bit slower than people want, but they are making progress year on year,” Gargan says. “It just doesn’t happen at the speed in which everybody would like it to happen overnight. It takes time to really make this all work together.”

Unclear expectations

One problem with IT leaders’ possible overconfidence about AI expectations is that most organizations have no concrete expectations to begin with, says Warren Wilbee, CTO of supply chain software provider ToolsGroup.

“Are the expectations a 10% productivity again, or a 2% drop in staffing?” he says. “The expectations are ill-defined.”

Other AI experts see AI enthusiasm outpacing the difficulties of deploying the technology. In many cases, company leaders underestimate the technology requirements and the compliance and governance demands, says Patrizia Bertini, managing partner at UK IT regulatory advisory firm Aligned Consulting Group.

“The pressure to ‘do something with AI’ has created a false sense of urgency,” she says. “Too many organizations jump in without a clear vision or understanding of what’s required to make AI work in practice. They’re focused on what to deploy, not how to deploy responsibly.”

The EU AI Act, for example, includes several regulations that many CIOs aren’t prepared for, she adds. “When we explain what’s needed under the forthcoming EU AI Act — from documenting data sources and bias testing to showing decision-making flows — we see jaws drop,” Bertini says. “Most CIOs admit they had no idea. Their compliance partners aren’t giving them the full picture.”

More work to do

In addition to compliance and governance challenges, many AI projects remain in the early stages, experts say. AI agents, in particular, show significant promise, but implementations are nascent, ToolsGroup’s Wilbee says.

Enthusiasm over AI isn’t misplaced, he adds, but the lack of enterprise-wide deployments comes from the scale of transformation required.

“While AI can be used as a feature upgrade, such as chatbots or similar tools, its true potential extends far beyond that,” he contends. “To fully realize its benefits, agentic AI can’t be treated as a simple plug-and-play solution — it demands rethinking workflows and reshaping how knowledge workers operate.”

Wilbee expects significant progress in a year as organizations come to grips with the operational shifts needed to deploy game-changing AI.

Many organizations’ leaders don’t understand the full implications of rolling out and using AI, he says, with many not realizing the extent to which the technology will change the nature of work. Instead of executing tasks, many employees will manage agents that complete those tasks — a seismic shift.

“Agentic AI holds enormous potential, but the path to full deployment will take time, requiring effort and investment,” he says. “Success will depend on how well, and quickly, organizations integrate the technology into their processes and establish the proper foundation around it.”

AI projects will still take time to reach mass deployment, adds Yoni Michael, CTO and cofounder of Typedef, an AI startup focused on turning protypes into deployments. While Michael agrees that enthusiasm is justified, he sees an inflection point where a large number of projects get to deployment coming in two to three years.

“There is real enthusiasm in the market — I don’t think most of these leaders are deluding themselves,” he says. “But what they often underestimate is the distance between ‘it works in a notebook or during a demo’ and ‘it works reliably, at scale, in production.’”

Guardrails needed

To move out of pilot purgatory, Michael suggests that CIOs invest in AI-native infrastructure as early as possible. “Traditional data stacks were never built for inference or unstructured data,” he says.

IT leaders should also embed accountability and guardrails into AI projects. They should tie pilots to business KPIs, and enforce predefined limits on errors, latency, and cost. CIOs should “ensure fallback paths if things misbehave.”

In addition, IT leaders should build cross-disciplinary teams, he recommends. “The future isn’t just data science; it’s data engineering plus ML plus systems plus reliability engineers working together,” Michael adds. “Foster collaboration between central IT, infrastructure, and AI teams. Don’t let AI be a black box — make it part of your core ops, not a side project.”

Finally, IT leaders should plan for continuous maintenance, not just a one-time launch, he says. “Models drift, data shifts, and usage patterns change,” Michael adds. “You need pipelines for retraining, rollback, shadow modes, and versioning.”