The AI ROI gap isn’t a model problem. It’s a workflow problem

Anthropic says Claude now writes more than 80% of the code merged at one of the most sophisticated AI companies on the planet. Foundry’s 2026 State of the CIO study says fewer than one in five enterprises can show that their AI initiatives have met or exceeded their ROI goals. Both numbers came out this spring. Both are true. And the distance between them is the most important thing an IT leader can understand about AI right now.

Because that distance isn’t a contradiction, it’s a lesson. And the profession sitting in the middle of it, software engineering, is the canary that explains why so much enterprise AI spend has produced so little measurable return.

The report everyone misread

When Anthropic published its recursive self-improvement piece, plenty of people read it as the starting gun for the job apocalypse. Claude writing its own code, models getting better at building models, humans narrowing toward oversight. If you wanted a headline about the end of the software profession, it was right there.

I read it almost the opposite way. What struck me wasn’t how far AI had come. It was how much had to be true first, even in the one profession built from the ground up to let it succeed.

I made this argument back in my “$570K canary” piece, and the Anthropic data only sharpens it. AI coding agents don’t work because coding models are special. The underlying large language models (LLMs) are the same ones answering support tickets and reviewing contracts. They work because software development already had the infrastructure that makes an agent’s output trustworthy: governance baked into branch protection and code review, observability through version control and CI/CD pipelines, evaluation through automated tests, persistent context through commit history. Developers built all of that for themselves over decades. They didn’t build it for AI. But it turned out to be exactly the scaffolding AI needed.

That’s the part the apocalypse reading skips. Claude’s coding gains are real. They also rode on decades of pre-built substrate. Both things are true at once, and the second one is the one CIOs should be paying attention to when it comes to gains from things like recursive self-improvement.

What the CIO data actually shows

Now hold that next to the State of the CIO numbers. Only 19% of the 662 IT leaders surveyed say their AI initiatives have met or exceeded business goals. Another 18% admit fewer than a third of their use cases are hitting defined expectations.

The easy explanation is that the technology isn’t ready. The data says otherwise. This isn’t for lack of trying, and it isn’t for lack of organizing. Eighty-three percent of respondents have stood up cross-functional steering committees or are about to. Just over half have some form of AI approval process in place, with another quarter building one. Forty-seven percent have formal success metrics, with a third more on the way. The field is pouring effort into the organizational machinery of AI. The ROI still isn’t showing up.

Here’s why I think that is. All of that machinery sits above the work. Steering committees, approval gates and KPI dashboards govern the org chart. But the value, or the leak, happens inside the workflow, at the level of the actual task the AI is doing. You can instrument your governance structure perfectly and still have nothing measuring whether the agent’s output was right at the point where it mattered.

TIAA shows how little the org chart settles. The firm is three years in, runs generative and agentic use cases across fraud detection and call centers, and has 85% of its people on TIAA Gate, its internal platform. It also has the full governance stack most CIOs are still assembling. None of it closed the gap. “You need to understand the full cost of operations,” its chief operating, information and digital officer, Sastry Durvasula told CIO.com, “the efficiencies of running tokens or how you’re handling traffic or RAG.” The structures were never the thing leaking value. The workflow underneath them was.

The barriers respondents named back this up. The top three are lack of in-house expertise (40%), ill-defined ROI metrics (32%) and murky corporate AI strategy (31%). Not one of them is “the model isn’t good enough.” And according to the full Foundry report, the expertise gap is deepest in healthcare (52%), retail (51%) and manufacturing (49%), the sectors whose core work looks least like a software development lifecycle. That’s consistent with substrate being the real variable, though a tighter market for AI talent in those industries is surely part of the story too.

The market is already voting

Look at where the AI is actually being pointed, and you’ll see enterprises sequencing by substrate even though nobody’s calling it that. Three-quarters of both IT leaders and line-of-business respondents say AI is primarily being used to automate internal processes rather than customer-facing applications.

That’s not timidity. It’s instinct pointing at the right thing. Internal processes are the ones with structured, observable workflows and users who tolerate a little friction. Customer-facing work is where the trust gaps are still wide open and the cost of a wrong answer is asymmetric. A bad internal draft gets fixed before anyone sees it. A bad customer answer is the whole ballgame.

I’ll be honest about a wrinkle in the data here, because a careful reader will catch it. The same study reports a near-mirror finding, that 66% to 69% of respondents say the bulk of their current AI work is customer-facing. The two stats sit a paragraph apart in the CIO study and almost certainly reflect how the question was framed rather than a real reversal. But the synthesis holds either way: even where customer-facing work is being attempted, it’s where ROI is least realized. The work that lands is the work with the substrate underneath it. The split only reinforces the point.

Sequence by readiness, not by ambition

So, here’s the prescription, and it cuts against the instinct most AI strategies are built on. Stop sequencing your AI portfolio by where the value looks biggest. Start sequencing it by where the work already has, or can be given, a structured workflow with a usable signal for whether the output was right.

The study itself shows what the alternative looks like. Andrea Ballinger, CIO at Rensselaer Polytechnic Institute, described the trap precisely. No one measures ROI on an ongoing basis, she said, “because we are facing counterpressures from every vice president and line-of-business domain looking to implement AI for their own optimization.” The result: “We are saying yes to everyone without stepping back and focusing on the business cases that show real value.” That’s value-led sequencing under pressure from every budget-holder in the building, and it’s exactly how you end up with a sprawling pipeline of pilots and a 19% success rate.

The counterexample comes from the same study. Thomas Prommer, a longtime CTO, CIO and CAIO, funds outcomes instead of deliverables. “We don’t fund ‘build a model,’ we fund ‘reduce returns by 8% on this category’ with checkpoints at 90, 180 and 270 days,” he explained. He kills any project that misses two checkpoints, “roughly a third of what we start, and that’s healthy.” Read that through the substrate lens and you see what he’s really doing. He’s manufacturing a correctness signal where the work didn’t come with one. He’s building the missing piece of scaffolding by hand.

That gives you a simple lens to run any candidate use case through. Does the work break into discernible stages? Can you observe what happens at each one? Is there a usable signal for whether the result was right? Score high on all three and you have a software-engineering-shaped problem, so go now. Score low and you have a choice: build the substrate first or wait. What you shouldn’t do is fund it at scale and hope the ROI materializes, because that’s the pile the 19% number is built on.

The hard part, and the honest caveat

Run the professions through that lens and they sort themselves. Finance is the closest cousin to software. Reconciliation, close processes, approval chains and audit trails already give you staged work with a clear “it reconciles or it doesn’t” signal, which is part of why financial services sits among the sectors furthest along with AI. Legal and medicine are harder. The workflow shell exists, intake to redline to filing, diagnosis to treatment to follow-up, but the correctness signal at the core is weak, delayed or confounded. You can automate the routine staged parts and you hit a wall at the judgment that defines the profession.

And that’s the caveat that keeps this honest. A structured workflow isn’t always buildable in software’s image. For the judgment core of some professions, the substrate is years out no matter how good the model gets or how mature your governance becomes. Anyone selling you a tighter timeline than that is selling.

But notice what this reframe does. It turns “our AI ROI is elusive” from a mystery you wait out into a sequencing-and-instrumentation problem you can actually test. Your timeline isn’t set by how smart the next model is. It’s set by how fast you build the substrate for your own domain, and that’s within your control.

The two numbers, reconciled

Put the 80% and the 19% back next to each other and they stop looking like a paradox. Software engineering didn’t win because its models were better than everyone else’s. It won because the work was already shaped to let an agent succeed, and the scaffolding that makes agent output trustworthy had been in place for decades before the agent showed up.

The question for the rest of the enterprise was never really whether AI can do the work. It’s whether your work is shaped so AI’s output can be trusted. That’s not something you wait for. It’s something you build.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?