Regrets set in for CIOs who deployed AI too soon

A vast majority of CIOs now regret major AI purchases their organizations have made, with many also being asked to defend AI outputs they can’t explain.

Three-quarters of CIOs say they have remorse over at least one major AI vendor or platform selection made in the past 18 months, with some of that disappointment driven by unexpected AI results, according to a survey commissioned by AI orchestration provider Dataiku.

Twenty-nine percent of CIOs say they’ve been asked to justify AI outcomes they could not fully interpret. Those numbers are concerning, says Kurt Muehmel, head of AI strategy at Dataiku.

“Reading that statistic, it makes me a little bit nervous being a member of society where corporations are increasingly running on these systems,” he says. “In the excitement to deploy this technology, which is understandable given the potential for it, we’re a step or two ahead of the governance frameworks.”

AI adoption is moving much more quickly than many other technologies, including machine learning in the last decade, Muehmel notes. With machine learning, there was a gradual building of capabilities, he says, and organizations collectively learned what works and what doesn’t.

“With AI, with agents especially right now, things have gone so quickly that we don’t have yet those full governance and agent ops capabilities in the way that we need,” he adds. “Those best practices aren’t well defined yet in the industry.”

The cost of switching

CIO regrets, meanwhile, seem to be tied to switching costs, Muehmel says. In some cases, companies that are launching agents are tying deployments to specific AI vendors, he adds.

“Imagine an agent, all the instructions, all the orchestration of how that is supposed to work,” he says. “If you too tightly couple that with one of your providers and then you realize that someone else came out with a better model or there’s a better framework, extracting that logic from that underlying system becomes super costly.”

The survey also suggests that CIOs feel heavy pressure to make AI work. Six in 10 say their CEOs have questioned their AI vendor or platform decisions in the past year, and 71% say it’s likely their AI budgets will be cut or frozen if targets aren’t met by mid-2026.

Some IT leaders say the survey’s results hit home. Early in his company’s adoption of AI, Tomas Kazragis, VP of engineering at email and SMS marketing technology provider Omnisend, was asked to explain outputs he couldn’t fully explain.

Like many other IT leaders, Kazragis was caught up in the AI hype and pushed too hard for results, he says. “There was a great deal of movement, but it was difficult to explain what outcomes we were actually aiming for,” he adds. “Basically, we asked people to move — and they did — without a clearly defined objective or measurable result.”

Kazragis still feels pressure to generate positive AI results, he says. “Competitors and professional networks are all buzzing about how AI will change the world,” he adds. “If you’re not in, you’re out. But when you observe all of this critically and keep a clear head, you quickly see the conversation is equal parts snake oil and genuine innovation.”

Moving too fast for regrets

It’s important for IT leaders to remain level-headed and not pressure themselves into following every AI trend blindly, Kazragis says. Still, he doesn’t have any remorse over any AI vendor or product decisions Omnisend has made.

“With the pace of innovation so rapid, we’ve replaced quite a few AI tools — but it’s not something to regret,” he says. “It’s the natural cost of working with innovative solutions, where the risk of replacement is inherently high.”

Lior Gavish, cofounder and CTO at AI observability vendor Monte Carlo, also doesn’t feel any regret over AI tools deployed.

“Some AI technologies proved highly successful, while others less so,” he says. “But experimentation and failures are essential in such a rapidly evolving space.”

Monte Carlo quickly learned that one key to successful deployment is connecting AI tools to the data they need, he adds. “We regretted the times we didn’t do it,” he adds.

Gavish does feel pressure to drive AI forward, he acknowledges. “The pressure to meet AI targets is coming from the rapidly evolving market,” he adds. “Our customers expect it, and our competitors will beat us if we don’t.”

Still, he sees the pressure evolving over time, if not lessening. After FOMO drove early adoption, organizations will shift from experimentation to accountability, he says.

“Enterprises are moving past pilots and asking harder questions about reliability, governance, and measurable ROI,” he adds. “The pace of adoption will continue, but the focus is turning from speed to trust and operational rigor, which is ultimately a healthier phase of the cycle.”

CIOs often get blamed for the consequences of the FOMO coming from above, adds Maya Mikhailov, CEO at fintech AI agent vendor SAVVI AI. Many AI decisions have been forced on tech teams by executives, she says.

“There is a massive disconnect between the AI demos and promises being sold to the C-suite and boards, and the reality on the ground about the enterprise’s data readiness for AI success,” she adds. “Unfortunately, the CIO bears the brunt of this disconnect because they are the tech guy who was supposed to make these choices work, even with legacy systems and broken processes.”

Companies should adopt AI based on specific needs and an understanding of their complexity compared to their value, Mikhailov adds.

“‘We need to buy ChatGPT and figure out what to do with it later’ is not a business strategy, but it’s unfortunately one that the CIO is now responsible for,” she says.