AI made software easy to build. Running it is the hard part

A few weeks ago, I found myself in yet another demonstration of an application that was not built by my team or my technology partners. The demonstration lasted 20 minutes and by the end of it, everyone in the room agreed that the application solved a genuine business problem. There was genuine admiration for the business team’s initiative, a few questions about future enhancements and the inevitable congratulations that accompany any successful AI story. Then somebody asked, almost casually, “Can IT roll this out quickly across the organization?” It was a short question that unfortunately needed a long and unpopular answer.

I am sure almost every technology leader reading this has experienced some variation of this moment. The application itself is rarely the problem. In fact, many of the applications I’ve seen over the past year have been remarkably good. What concerns me most is something rather different. Somewhere along the way, we’ve started confusing the act of building software with the responsibility of running it. AI has dramatically reduced the effort required to create an application, but it has done little to reduce the effort required to own and operate one sustainably.

That distinction may sound subtle, but it is not. I think this might be one of the most significant leadership challenges of the AI era.

The success nobody planned for

None of this should come as a surprise. For years, we have been telling the business to become more digitally savvy. We invested in low-code platforms and citizen development initiatives, organised hackathons and innovation challenges, and repeatedly argued that technology shouldn’t become a bottleneck to solving business problems. The rapid emergence of AI coding assistants has simply completed that journey. Today, anybody from finance, marketing or operations can turn an idea into a working application faster than most technology teams can schedule a requirements workshop.

Personally, I think that’s amazing. Some of the most interesting ideas I’ve seen this year didn’t emerge from tech teams. They came from people who understood the business problem intimately and no longer needed permission to begin experimenting. That’s a future I would much rather embrace than resist.

The problem isn’t that business teams are building software. The problem is that successful prototypes have a habit of raising enterprise expectations quickly. Yesterday it was a departmental experiment. Today it is being demonstrated to the executive committee. Tomorrow someone is asking why the rest of the organization isn’t using it. The application hasn’t changed; the expectation has. In many ways, this feels like the next evolution of what we’ve traditionally called shadow IT. The difference is that these applications are often better engineered, solve genuine business problems and, ironically, are being built with the very experimentation that technology leaders have spent years encouraging.

The thin line between building and running

This is the point where discussions between business leaders and technology leaders begin to fall apart. The business sees an application that works. Technology sees an application that now needs to survive outside the protected environment in which it was created. Those are fundamentally different things.

A prototype rarely worries about things like identity management, resilience, audit trails, backup policies, API versioning, support models or regulatory obligations because none of those questions matter while an idea is still being tested. However, they become important once the organization decides that the application has graduated from an experiment to an enterprise capability. In many ways, this is the point where software engineering gives way to software operations—a discipline that organizations like Google have spent years formalising through practices such as Site Reliability Engineering (SRE).

This is the point where many organizations are beginning to underestimate the challenge. AI has democratised software development. It has not democratised enterprise operations. Running software is an entirely different discipline. It is less visible, less celebrated and considerably less exciting than building it, but it is also the reason enterprise technology exists. Every application that enters production quietly accumulates obligations. Someone has to secure it. Someone has to integrate it. Someone has to monitor it, patch it, support it and explain it to an auditor. Eventually, someone has to retire it. None of those responsibilities disappear simply because the first version happened to be created in forty-eight hours using an AI.

Walking the tightrope

The temptation for technology functions is to respond in one of two ways. The first is to become the organization’s brake pedal. Every application must now navigate governance committees, architecture reviews, security assessments and operational checklists before it is allowed anywhere near production. The enterprise is undoubtedly safer, but enthusiasm evaporates quickly when innovation feels like it needs intricate planning and convoluted permissions.

The second temptation is more subtle, and in many ways more dangerous. We become so determined not to discourage innovation that every successful prototype quietly becomes another production application. We congratulate ourselves on enabling the business while gradually accumulating a software estate that nobody really owns or understands. Six months later, the original creator has moved to another project, the AI prompts have disappeared, users have doubled, integrations have multiplied and suddenly the technology team is required to support something it neither designed nor approved.

Neither extreme is sustainable.

This, I suspect, is the balancing act that leadership will increasingly be judged on. Not whether we can prevent people from building software—that battle has already been lost, and rightly so—but whether we can encourage experimentation without allowing enthusiasm to become tomorrow’s operational burden.

I’ve discovered that the tone of these conversations changes entirely if we begin with curiosity instead of governance. Rather than asking why technology wasn’t involved earlier, we now ask what problem the team was trying to solve. It sounds like a small change, but it transforms the discussion. People become far more willing to talk about security, resilience and operational ownership once they know those questions are intended to preserve what they’ve built rather than prevent it from succeeding.

Stewardship, not gatekeeping

Interestingly, this isn’t simply a challenge that individual technology leaders are experiencing. Recent research points in the same direction. The 2025 DORA State of AI-assisted Software Development report concludes that AI acts primarily as an amplifier. It magnifies the strengths of organizations with mature engineering practices and exposes the weaknesses of those without them. In other words, the greatest returns from AI don’t come from the coding tools themselves, but from the quality of the underlying engineering and operational system.

Gartner arrives at a similar conclusion from a different perspective. In its analysis of enterprise AI coding agents, the firm argues that the market is rapidly evolving beyond developer productivity towards operational excellence and enterprise readiness. As organizations begin operationalising AI-generated software at scale, governance, operational ownership and long-term lifecycle considerations become just as important as the tools themselves.

None of this should really surprise us. We’ve spent the last couple of years asking whether AI can help us build software faster. That question has largely been answered. The more interesting question now is whether organizations are prepared for the consequences of making software creation almost frictionless. Every successful application creates an obligation that lasts far longer than the weekend it took to build.

Technology leaders have traditionally thought about technical debt as ageing platforms, deferred upgrades, architectural compromises and code that has outlived its original design. CIO.com has written extensively about the long-term business impact of technical debt. I believe AI is quietly introducing another form of debt that deserves equal attention—operational debt.

Operational debt begins the moment an application is promoted from a successful prototype to a business-critical service without a clearly defined operating model. Every application that is enthusiastically pushed into production quietly becomes another long-term obligation. It needs monitoring, support, ownership, governance, funding, documentation and, eventually, retirement. Unlike technical debt, operational debt is rarely visible until something fails, an audit raises uncomfortable questions or the person who originally built the application has long since moved on.

The role of leaders may no longer be to decide who gets to write software. AI has already democratised that capability. Our responsibility is something altogether more nuanced. We have to preserve the excitement, curiosity and initiative that AI has unlocked across the business while ensuring that the enterprise remains secure, resilient and supportable. Push too hard and we become the one that quietly kills innovation. Push too little and we inherit an estate of applications that nobody is truly prepared to operate.

This isn’t a governance problem. It’s a leadership one…and I suspect it may well become one of the most defining responsibilities of enterprise technology for years to come.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?