CIOs must mind their own data confidence gap

As far as many C-suite business and IT executives are concerned, their company data is in great shape, capable of fueling data-driven decision-making and delivering AI-powered solutions.

But the closer an IT leader is to that data, the less confidence they have in its quality, according to a recent survey from IT consulting firm Softserve, which found that nearly half of C-level execs at large enterprises, including C-level IT leaders, believe their organization’s data is fully mature, while just 37% of director-level data and AI leaders see the same.

This data confidence gap between C-level executives and IT leaders at the vice president and director levels could lead to major problems when it comes time to train AI models or roll out other data-driven initiatives, experts warn.

Moreover, 68% of vice presidents in charge of AI or data management already see their companies making decisions based on bad data all or most of the time, versus 47% of C-level IT leaders.

It’s not surprising to see the differences when C-level executives tend to receive PowerPoint-level snapshots of IT problems, including data quality, says Timothy Bates, a professor in the College of Innovation and Technology at the University of Michigan 

“Executives see dashboards — clean, aggregated, polished,” Bates says. “Directors see the backend — broken pipelines, inconsistent definitions, alerts missing context.”

Bates, the former CTO at Lenovo and General Motors, saw vastly different perspectives on IT problems at the giant vehicle maker, with director-level IT leaders often pointing out issues that those in the C-suite couldn’t see.

“The directors weren’t being pessimistic; they saw the gaps dashboards don’t show,” he says. “Directors are often more accurate in their confidence assessments, because they’re swimming in the systems, not just reviewing summaries.”

Data strategies in the balance

In addition to a data visibility gap between levels of IT management, quality problems often come from piecemeal IT infrastructure, with many companies using multiple IT vendors’ products to achieve desired functionality, says Anant Agarwal, co-founder and CTO at Aidora, developer of AI-powered HR software.

To run an internal tool, a company may use cloud services from one vendor, database APIs from a second vendor, a caching service from a third vendor, an AI tool from a fourth, and a sign-in service from a fifth, he says.

“Essentially, multiple pieces of smaller software owned by different vendors are all coming together to build the product,” he adds. “All these systems generate their own data, and conscious effort is required to make all the generated data flow into a centralized data lake with appropriate annotations so that permissions can be figured out.”

Then, after the internal service is finished, IT teams move onto the next thing, Agarwal says.

“As soon as the functionality is built out, teams mark the project as complete and send announcement emails, and there is no incentive to ensure that all the data is flowing through properly to allow for future usage of data,” he adds. “Teams tend to prioritize short-term wins over a long-term outlook.”

That emphasis can erode an organization’s data foundation over time. Lack of adequate funding for data management strategies and an emphasis on digital over data initiatives are just a few of the other issues derailing data-driven projects at most organizations.

The rush to AI

Data quality problems have been compounded in the past two years, as many companies rushed to adopt gen AI tools, says Rodion Myronov, Softserve’s assistant vice president for big data and analytics.

“When the board asks the C-level about this technology, naturally people feel obliged to come up with something,” he says. “You can’t really say, ‘No, I don’t know what we can do with that.’”

Many companies started gen AI projects without defining the problem they were trying to solve and without cleaning up the data needed to make the projects successful, he says. In some cases, internal data is still scattered across many databases, storage locations, and formats.

When companies launch AI projects without collecting and cleaning up their data, they open themselves up to AI hallucinations that can lead to huge business missteps, says Ram Palaniappan, CTO at IT solutions provider TEKsystems.

“If I have AI-based revenue reporting and put it in front of the boardroom, and for some reason, I miss one region, people are going to have less trust in your report,” he adds.

The initial public relations hype surrounding gen AI led to many rushed projects that failed to use the right data, he says. “The AI PR is catching up with the C levels and the VP level,” Palaniappan adds. “When it comes to the actual execution, the directors and the managers who are on the front lines of execution are seeing all of these gaps, and they are now calling it out.”

Problems with bad data

A lack of good data can lead to several problems, says Aidora’s Agarwal. C-level executives — even CIOs — may demand that new products be built when the data isn’t ready, leading to IT leaders who look incompetent because they repeatedly push back on timelines, or to those who pass burden down to their employees.

“The teams may get pushed on to build the next set of things that they may not be ready to build,” he says. “This can result in failed initiatives, significantly delayed delivery, or burned-out teams.”

To fix this data quality confidence gap, companies should focus on being more transparent across their org charts, Palaniappan advises. Lower-level IT leaders can help CIOs and the C-suite understand their organization’s data readiness needs by creating detailed roadmaps for IT initiatives, including a timeline to fix data problems, he says.

“Take a ‘crawl, walk, run’ approach to drive this in the right direction, and put out a roadmap,” he says. “Look at your data maturity in order to execute your roadmap, and then slowly improve upon it.”

Companies need strong data foundations, including data strategies focused on business cases, data accessibility, and data security, adds Softserve’s Myronov.

Organizations should also employ skeptics to point out potential data problems during AI and other data-driven projects, he suggests.

“If you look at any POC [proof of concept], not even talking about AI, and the team running it, you will find people who are invested, who believe that the result will be achieved,” he says. “You will not find people who play the role of QA in software development, people who are trying to find the imperfections.”

Without an employee in the role of a POC skeptic, companies can spend too much time and money on projects that ultimately fail, he says. Failing fast is a better outcome.

“There are developers who mostly see sunny-day scenarios,” he says. “Good developers are focused on building something that works, but there should be a completely separate role that focuses only on finding something that does not work.”