AI agents are turning data silos into an existential infrastructure problem

Enterprises have built their data systems for humans, but AI agents need a whole new infrastructure.

Separate research from Cloudera and Google/MIT found that, not surprisingly, there is fervent enterprise interest in AI agents, but underlying infrastructure struggles to keep up. Deployments continue to be hampered, sometimes even abandoned, largely due to issues with data access, context, and governance.

“Enterprise adoption of agentic AI is on the cusp of an extraordinary acceleration,” the Google/MIT report noted. “As organizations look to scale agentic AI across the enterprise, they cannot ignore their data systems.”

Resolving data bottlenecks, then, should be an immediate priority.

Projects delayed, inaccessible data

Cloudera’s report, created in partnership with Wakefield Research, describes the need for a “great AI re-architecture.”

Of the 1,500 enterprise architects and cloud infrastructure leads surveyed, a stunning 95% said they had delayed or cancelled AI projects, in some cases six or more, in the past year, due to issues with data governance, compliance, or regulatory issues.

A wide majority also reported that AI integrations have changed their data storage and architecture practices, AI workloads have increased infrastructure costs, and current data architecture requires a “significant overhaul” to meet AI goals.

“Even if enterprises are ready to use AI, many are coming to the realization that the foundational infrastructure it relies on is not,” the report noted.

Similarly, more than half of the 300 IT execs and heads of product, IT, data, and AI responding to the Google/MIT survey said they have paused or delayed the deployment of AI agents to address foundational data issues, such as siloes or lack of context. Further, more than half reported that legacy data systems are preventing them from scaling agentic AI and are having a “significant negative impact” on their AI ROI. High latency has also hampered AI from making decisions at “high velocity.”

“To make good decisions and take effective action, agentic systems need a data foundation that is multimodal, context-aware, and instantly available,” the report noted. “Legacy data systems struggle to meet these demands, ultimately compromising AI trustworthiness.”

Google and MIT identified several reasons that enterprises struggle to deploy AI agents, most notably:

• Entrenched siloes: Data sits in disconnected systems, or is “pocketed away” in different departments with no integration layer; it could also be in outdated formats, old management platforms or logs, or on obsolete IoT devices.
• Difficult-to-access data: “Dark” or unstructured data is contained in different formats like images, video, or PDFs.
• Insufficient access to real-time data: Legacy batch processing architectures can make in-time action a challenge.
• Lack of context: Agents often receive basic metadata rather than enterprise-specific semantics, so they struggle to make relevant connections or suggestions.

“Without this deep understanding, data cannot be highly relevant to specific use cases,” the report noted.

‘Data leaders’ versus ‘data laggards’

This is not to say that enterprises aren’t deploying AI; quite the contrary. Nearly all respondents (98%) to the Google/MIT survey are already using agentic AI or plan to soon. One in 10 is using it widely and nearly three quarters have deployed it in a limited fashion.

The most common uses for AI agents right now are in customer service (routing requests and resolving issues), IT systems management (managing user access and incident response), and IT security (anomaly detection and threat scanning). In the near future, the survey said, enterprises also plan to use AI agents in HR, finance, and supply chains.

“It is easy to understand why companies are eager to put AI agents to work,” the report noted. Agents can supercharge productivity and efficiency so employees can turn to more strategic work.

The enterprises seeing the most success are what Google and MIT refer to as “data leaders,” those that give AI systems access to more than 70% of their data. “Data laggards,” by contrast, share just 30% or less of their data with AI. Interestingly, 100% of data leaders say their agents make “mostly” or “consistently” accurate and relevant decisions, while just 22% of data laggards say they have that trust.

Agents need what the report calls “frictionless access” to operating systems, multimodal data, and context, helping them understand how data maps to different teams’ goals. “The most important initiative to enable scaling among all respondents is improving access to structured and unstructured data for AI agents.”

To successfully scale agents, the report recommended that enterprise leaders prioritize several data initiatives. First, improve access to data; discover, inventory, and classify data, both structured and “dark”/unstructured. After governance principles are applied, information can then be extracted and connected to AI agents.

Next, “put a premium on context” by giving agents enterprise-specific data. Replace batch processing with streaming and event-driven pipelines as well.

Finally, think AI-native. “AI-native systems are designed to operate in an AI environment and built from the outset to leverage AI for data management and decision-making,” the report noted.  

AI needs multimodal cloud environments

AI needs a lot of data that is often spread across on-premises systems and cloud, SaaS, and edge environments. In fact, 97% of respondents to Cloudera’s survey said they move data between environments monthly, and nearly one-third do so daily.

However, nearly three-quarters (73%) said AI integration makes data governance more complex.

“Enterprises had good governance systems when humans were the only ones accessing data manually,” the report noted. “But when thousands of agents provide support to employees, customers, or partners the story changes and governance requires another dimension.”

This makes private AI and data sovereignty critical, Cloudera said. Enterprises should have a data foundation that is unified and provides control over 100% of organizational data, wherever it resides. They must also think about where AI workloads run, while still maintaining control over sensitive data and keeping cost controls and flexibility in mind.

One notable trend Cloudera uncovered is a “resurgence” of on-premises and private cloud environments. Over the last 12 months, 66% of respondents moved AI workloads from public cloud back to on-premises or private cloud environments. And 25% said they plan to place more emphasis on a hybrid-first approach, 24% said they are planning to increase on-premises spend, and 22% plan to increase edge spend.

“IT leaders are putting more emphasis on moving workloads to the environments that they are best suited for, based on performance, cost, latency, governance, availability, and accessibility,” the report stated.