Fast vs. slow: the real impact of AI adoption speed

There are a lot of reasons to take a slow and careful approach to generative AI. The technology is changing quickly, so investing a lot of money in the wrong platform could end up costing a lot of money.

Generative AI still has accuracy and safety problems, and the copyright issues haven’t yet been settled in the courts, all of which could create legal liabilities or other problems. And, of course, many early projects will fail to offer any actual business value, making them a waste of time and resources.

According to a September IDC survey, 70% of CIOs reported a 90% failure rate for their custom-built AI app projects, and two-thirds reported a 90% failure rate with vendor-led AI proof-of-concepts. And Rand Corp. puts the AI failure rate at over 80%.

However, some early adopters report revenue growth, productivity enhancement, and early efforts bearing fruit by helping companies develop critical skills and abilities related to gen AI. The Boston Consulting Group, in fact, says companies that have adopted AI early claim 1.5 times higher revenue growth than other companies. So how do you reconcile the high failure rates of AI projects and reports of business benefit by early adopters? Both of these things can be true. Early adopters will try many different approaches before they find ones that work, and the ones that work will be scaled up, put into production, and deliver value to enterprises.

Build versus buy

Technology vendors are rapidly adding gen AI capabilities to all their products and services. But some companies can’t wait that long. Intuit, for example, has built an agentic AI system to help business owners get paid 45% faster. “It helps business owners understand what the invoices are, when to send out reminders, and how to collect money,” says Ashok Srivastava, Intuit’s chief data officer. To do this, the company built its own gen AI operating system. “It abstracts away the complexity of the platform so developers can develop on it,” he says.

GenOS was launched in June 2023 and this past September, it was augmented with the GenOS AI Workbench, a dedicated development environment. Intuit has also built an orchestration layer for agentic workflows, a set of security, risk, and fraud guardrails, a user experience framework with more than 140 components, widgets and patterns, and a model garden of leading commercial and open-source LLMs, plus Intuit’s own custom-trained domain-specific models.

“We’re ahead of the platform players by 18 months to two years,” he adds.

So what happens when Microsoft, Google or AWS rolls out their own operating system for gen AI? “As they catch up, we move onto the native services they offer,” Srivastava says.

So, in a sense, all this work would have been wasted; at some point, the company will switch to whatever the major vendors come out with. But, until then, it’ll be able to reap the benefits of its early investments.

“That’s how we stay ahead,” he says. “We can’t wait. We must build technology and, as we do this, the platform is continuing to evolve. Sure, there are services we shift from our own capabilities to others’. What I can tell you is these are fundamental investments we’re making in order to drive business forward.” For example, the new gen AI capabilities have resulted in 15% average productivity increase and 30% faster coding times.

Another company building its own agentic AI framework is Capgemini. “Gen AI has been transformative,” says Jiani Zhang, EVP and chief software officer at Capgemini Engineering. “There’s huge potential, specifically in software engineering.”

So in May last year, the company began building its own agentic framework. “We built something in-house because we wanted to be more open source so we’d be more adaptable,” she says. “And we started utilizing it in July and now it’s very robust.” The platform is composed of many different specialist agents to migrate old code, she says, with an agent that can generate code, an agent that can build requirements, and an agent that can build architecture. And there are specialist agents for particular purposes, such as for working on code for automotive software.

There’s also an orchestration layer that allows all the agents to talk to each other, a way to keep track of different iterations of the code, and self-diagnosis capabilities.

At the beginning of 2024, gen AI was all about individual use cases along the software lifecycle, Zhang says. Today, it’s about looking at code much more holistically. That’s a very rapid pace of change. “You can’t take the technology based on where it was in March,” she says. “From an adoption perspective, you can’t stand still.”

Intuit and Capgemini aren’t alone in taking an aggressive approach to gen AI deployments and innovation. According to a recent survey of 2,500 senior leaders of global enterprises conducted by Google Cloud and National Research Group, gen AI leaders are those companies that have four or more use cases in production and have invested more than 15% of their total operating expenses in gen AI over the previous year.

In addition, 69% of leaders use gen AI for at least half of their core functions, compared to 36% of other organizations. And they’re seeing returns. In the survey, gen AI leaders are 33% more likely to report revenue increases of 10% or more driven by gen AI, and see substantial efficiency gains as well, reporting ROI for gen AI projects related to improving back office processes, individual productivity, engineering and developer productivity, and sales and marketing.

Moving fast by leveraging commercial platforms

Not all AI leaders build everything from scratch in order to move fast, though. Take for example RSM, a global accounting firm with around 20,000 employees.

“All the data providers like Dun & Bradstreet, and software providers like ServiceNow, are embedding AI into their products,” says Sergio de la Fe, RSM’s enterprise digital leader. “We’re working to understand what they’re developing. I don’t want to waste my money if one of my partners is going to be building this. We’d rather invest our money in areas where we have domain expertise. I don’t want to go fast and build something I’m going to throw away. That’s not smart.”

RSM put together an AI steering committee in 2023 and identified four main types of use cases that were critical to its business: chat, document creation, document evaluation, and data analysis.

“These four main themes represent hundreds of use cases,” de la Fe says. “And we started with the use cases, not with the technology. We believe the technology supports the use cases, not the other way around.”

The company decided on OpenAI running on a private Azure cloud. “We’re a Microsoft shop,” he says, “so we’re on the Microsoft Azure platform and it was simple for us.”

For its automated compliance system, RSM fine-tuned an OpenAI model, mapping government regulations around the world to client internal controls and making recommendations. Today, there are a few dozen different use cases in production, and a couple of dozen more in development. “We’re moving very fast to put these into a queue to execute on, pilot, and test,” he says. So far, the biggest benefit has been improved efficiencies and increased quality.

“It’s about allowing our professionals to spend more time doing the value-added parts of their job,” he adds. “If we’re able to reduce the time spent on the mundane tasks, and allow more time for increased quality, that’s incredibly important.”

Another organization that’s accelerating its adoption of gen AI by leveraging vendor capabilities is payments company ACI Worldwide. With over 3,500 employees, everyone there has some type of gen AI at their disposal, whether for emails, or summarizing Teams meetings. For its internal knowledge base, it uses a fine-tuned version of OpenAI’s ChatGPT. “We’re definitely not going slow,” says Abe Kuruvilla, the company’s CTO. But it’s also not building all its gen AI platforms in-house. For code generation, for example, the company is using GitHub Copilot. “We’re getting comfortable with it,” Kuruvilla says. “Still, our senior engineers look at the code before we check it in.”

Today, 50% of the engineering staff is using it, and the company hopes to get to two-thirds this year. For general employee productivity, the company uses Microsoft’s Office 365 Copilot. It’s also a Salesforce customer, and is looking at deploying the agentic functionality that’s becoming available in the platform.

The main strategy, Kuruvilla says, is to deploy gen AI for internal use cases first, with humans in the loop to validate quality and outcomes. “It’s about impact and making sure you’re identifying the right use cases,” he says.

The case for education

AI leaders are reporting clear financial benefits to their gen AI deployments, but the promise of early adoption is more than just short-term ROI.

“There’s so much learning that happens through experimentation and iteration,” says Gartner analyst Arun Chandrasekaran. “You need to have a growth mindset when it comes to AI. You have to be actively experimenting.”

That doesn’t mean companies should randomly do AI projects, he adds. Organizations need to have clear goals in mind, but need to be prepared to not get it right on the first attempt.

“You have to learn from the failures and iterate,” he says. “Failure is not a stigma. Failure is part of the learning path. And this is true for life in general. If you succeed at the first attempt, it just means you set the bar too low for yourself.”