AI data readiness: C-suite fantasy, big IT problem

Business leaders may be confident that their organizations’ data is ready for AI, but IT workers tell a much different story, with most spending hours each day massaging the data into shape.

Nearly nine in 10 business leaders say their organizations’ data ecosystems are ready to build and deploy AI at scale, according to a recent Capital One AI readiness survey. But 84% of the IT practitioners surveyed, including data scientists, data architects, and data analysts, spend at least one hour a day fixing data problems.

Seventy percent of those IT pros spend one to four hours a day remediating data issues, while 14% spend more than four hours each day, according to the survey.

The survey points to a fundamental misunderstanding among many business leaders regarding the data work needed to deploy most AI tools, says John Armstrong, CTO of Worldly, a supply chain sustainability data insights platform.

“There’s a perspective that we’ll just throw a bunch of data at the AI, and it’ll solve all of our problems,” he says. “It says our job as technology leaders can help educate our audience on what is possible and what it will take to get to their goal.”

The implications of the ongoing misperception about the data management needs of AI are huge, Armstrong adds. When he talks to other IT leaders, they all are struggling with pressure to adopt AI, Armstrong says.

“It’s a big, big issue, because if not done right, your organization could spend literally millions of dollars on the wrong solution set to achieve the wrong outcome,” he says.

Misunderstanding the power of AI

The survey highlights a classic disconnect, adds Justice Erolin, CTO at BairesDev, a software outsourcing provider.

“Often, executives are thrilled by the promise of AI — they’ve seen it shine in pilots or presentations — but they don’t always see the nitty-gritty of making it work day-to-day,” he says. “That’s where the friction arises.”

Confidence from business leaders is often focused on the AI models or algorithms, Erolin adds, “not the messy groundwork like data quality, integration, or even legacy systems.”

Successful pilot projects or well-performing algorithms may give business leaders false hope, he says. “The bigger picture can tell a different story,” he adds.

For example, one of BairesDev’s clients was surprised when it spent 30% of an AI project timeline integrating legacy systems, Erolin says.

While initial work to fix data problems should be expected before an AI project, ongoing repair of data problems taking hours of staff time per day can be a warning sign that the organization’s data wasn’t ready for AI, Erolin adds. Organizations ready for AI should be able to automate some of the data management work, he says.

“If you’re spending so much time to keep the lights on for operational side of data and cleansing, then you’re not utilizing your domain experts for larger strategic tasks,” he says.

The legacy problem

Legacy systems that collect and store limited data are part of the problem, says Rupert Brown, CTO and founder of Evidology Systems, a compliance solutions provider. In some industries, companies are using legacy software and middleware that aren’t designed to collect, transmit, and store data in ways modern AI models need, he adds.

“Data quality is a problem that is going to limit the usefulness of AI technologies for the foreseeable future,” Brown adds. “Legacy systems that have limited input data fields or are forced to recycle account numbers are still prevalent in the industry, which also give rise to corrections which AI cannot fathom.”

To fix the problem of too-high expectations paired with low data readiness, CIOs and IT leaders should look to transparency and collaboration, Erolin says. BairesDev has focused on educating non-technical stakeholders about the realities and challenges of AI implementation, he says.

“When executives understand the real challenges — and the time tech teams spend fixing them — they’re more likely to invest in robust data practices and align expectations,” he says. “It’s all about getting everyone on the same page​​.”

While there seems to be a disconnect between business leader expectations and IT practitioner experiences, the hype around generative AI may finally give CIOs and other IT leaders the resources they need to address longstanding data problems, says Terren Peterson, vice president of data engineering at Capital One. The financial services company commissioned the survey because of its own interest in deploying AI tools to serve its customers, he adds.

“Data hygiene, data quality, and data security are all topics that we’ve been talking about for 20 years,” Peterson says. “There’s that part of me that thinks of it like, ‘Hey, is AI and ML going to be a catalyst that’s going to start to raise the attention of some of these sort of data foundational elements?”

The AI revolution may drive an understanding that data quality is important, he adds. “Even though it’s maybe been in the backlog of different CIOs agendas, it’s now going to raise in priority.”

Small prototypes to the rescue

While many business leaders are focused on deploying gen AI because of the current hype, Wordly’s Armstrong recommends IT leaders focus on use cases instead of specific AI technologies. In some use cases, older AI technologies, such as machine learning or neural networks, may be more appropriate, and a lot cheaper, for the envisioned purpose. Gen AI uses huge amounts of energy compared to some other AI tools, he notes.

He also recommends that CIOs launch small prototypes to find the best AI use cases for their organizations, with a recognition that some of the experiments won’t work out.

“Experimentation doesn’t have to be huge, but it breeds familiarity,” he says. “It starts to inform the art of the possible. If I had to give one piece of tactical advice, it would be like a slow burn, consistent investment and not productizing.”

Innovation often involves a lot of misfires, he adds.

“You want to build up a set of knowledge,” Armstrong says. “Everybody wants iterative, fast fail, fast development, but nobody wants to fail. And it’s such a hypocrisy in our space. Try it, and if it works, you want it, and if it doesn’t work, you learn.”