5 mistakes tech leaders make when deploying enterprise AI

The modern CIO has perhaps the hardest iteration of the job: transforming enterprises that run on SAP, ServiceNow and even fax machines into “AI-native,” “AI-first” organizations.

From choosing between hundreds of platform options, to receiving yearlong timelines to spin up a few chatbots, to employees who just keep feeding their team’s internal data to ChatGPT, dozens of CIOs feel stuck between a rock and a hard place.

But the pressure to increase operational efficiency grows each day, with 90% of enterprises actively adopting AI agents, and 79% of enterprises expect full-scale adoption of agentic AI in the next 3 years.

It can feel impossible to know what to avoid and where to look more closely, on the path to enterprise AI transformation, which is why I’ve curated the top missteps I’ve noticed from working with hundreds of enterprises over the past few years to implement AI solutions.

They’re hard-won insights, and I hope they’ll be useful.

1. Starting with the wrong use cases

Often, CIOs are looking to show the board something impressive, so they greenlight an ambitious marketing or sales AI project that ends up taking six months and never quite works the way anyone hoped. Once this happens, teams get demoralized and skepticism for AI stealthily increases.

The better approach is almost always to start small. Pick something repetitive, like compliance documentation, IT ticket routing, HR policy questions and onboarding checklists. These tasks are the perfect, deeply unglamorous proving ground. They’re well-defined processes, they happen constantly and the ROI is easy to measure (more on that next!).

You’ve heard the statistic that 95% of AI pilots are failing. Regarding the MIT study, Forbes also reports that “the bulk of investment (roughly 50% to 70% of AI budgets in executive samples) has flowed to sales and marketing pilots. These projects are easy to pitch internally…but the real cost savings are emerging in back-office functions.” Again, we say, start small.

2. Not measuring ROI

I’ve seen a lot of AI projects get launched with good intentions and zero accountability: no baseline, no tracking and six months later, nobody can say whether it worked. That’s a serious problem for sustained investment in AI and the enterprise of the future, and for building a culture where AI is taken seriously.

My team always recommends a structured proof-of-value period. Three months is the sweet spot—long enough to get an encouraging signal, but short enough to stay focused.

Going in, CIOs should clearly define what success looks like in tangible terms of hours saved per week, reduction in processing time and operational cost savings (if you can get there). At the end of 90 days, you should be able to stand up in front of your executive team and say exactly what you got for your investment, even quantifying in terms of full-time employees.

Quantitative discipline is what separates organizations that have the focus to scale AI from ones that may keep running pilots indefinitely.

3. Letting engineers be the bottleneck

Let’s start with the facts: you no longer need a software engineer to build a functional AI agent. The no-code and low-code AI builder company landscape has blossomed; new platforms seem to appear every day. The industry has matured to the point where the compliance analyst, the operations manager and the HR business partner can build and iterate on their own solutions.

I’ve spoken to CDAIOs who really struggled before democratizing building in this way. Abundant ideas from every team for the next great AI agent would be routed through engineering. Soon, a month-long queue was the only thing in sight. All of this changed when leadership realized that having a vast network of builders, with a layer of technical support to bring agents to their final form, is a massive competitive advantage.

Of course, the risks are high if employees start building on their own without insight into audit trails, version control, secure connections with internal databases or tools and more. But the trick is to find balance so that the people closest to the problem can ultimately feel empowered to build the very solution that will save them hundreds of hours in the long run. In the AI era, it’s the CIO’s job to give their teams permission to learn and experiment, without compromising on security and privacy.

4. Putting AI agents outside of where people work

This one is a silent killer: Sometimes, someone builds a great tool, deploys it as a standalone chatbot or an API that only works sometimes, and then wonders why adoption is flat. The answer is almost always friction. Workflows are sticky, and processes become habits over time.

All to say that interfaces matter enormously. An AI assistant embedded directly in SharePoint and a personal AI assistant that lives inside Slack or Microsoft Teams can meet people where they already work. Protocols like MCP are making this even more powerful, meaning AI workflows can run directly from tools like ChatGPT or Claude without any custom integrations.

When we prioritize interface design and integration, adoption rates can radically improve. When we ignore it, we end up with cutting-edge tools that collect dust.

A related note: Discovery is also key. Something I’m seeing increasingly is enterprises that have their own internal “app stores” for AI agents, organized by folders for categories and teams. Each user can favorite and search this repository, and both help prevent duplicate work and boost adoption and exploration.

5. Underestimating how much AI is going to transform the enterprise

Some CIOs are still treating AI as an efficiency tool, merely a way to shave time off existing processes. It dramatically undersells what is coming. This is the agentic moment. Most don’t realize that browser and computer use capabilities mean that AI can operate inside legacy platforms and interfaces that were previously unreachable by automation. Already, this technology can already handle a wide range of enterprise tasks from end to end, completely and consistently.

Our world has changed, and we foresee that the modern enterprise will be one where AI agents work alongside human employees as genuine contributors.

The future will reward the CIOs who take a long-term view at this time, thinking now about the challenge of scale. If—and when—you have hundreds of people building AI agents across your organization, you need version control, deployment pipelines and approval processes not for traditional software, but for AI workflows. You need role-based access controls that account for what AI agents can see and do, project-level controls and an end-user connection check to ensure that agents interacting on behalf of humans only see data that they truly have access to. The CIOs who are aware of the need for rigorous and new methods of governance have a meaningful head start on scaling AI across the entire enterprise.

The door is open

You don’t need to be the CIO with the largest team or the most technically sophisticated to achieve success as you begin your AI deployment. Just by starting with a clear use case, measuring results honestly, getting the right people building, deploying tools where people work and taking the long view on AI, you can get further than many.

The enterprise of the future awaits. Are you ready?

This article is published as part of the Foundry Expert Contributor Network.
Want to join?