Has agentic AI outgrown the data organization?

Recently, I participated in an architecture review for a Voice AI initiative. The initial proposal was heavily centered on the data required to provide context for the agent. The discussion focused on retrieval mechanisms, customer history, and knowledge access patterns. But as the review progressed, the discussion quickly went beyond data. Questions emerged around identity and authentication flows, telephony integration, cross-channel continuity, quality assurance of interactions, escalation handling, operational policies, and how operational knowledge management systems would contribute to the agent’s reasoning context.

What struck me was how a discussion that initially focused on AI and data requirements gradually revealed something much larger: Agentic AI systems do not operate purely within the boundaries of data systems. They sit at the intersection of data, applications, operations, governance, security, and organizational knowledge.

For years, most enterprises have neatly bundled “Data & AI” under one umbrella. That structure served its purpose well during the early days of machine learning and deep learning, when success depended heavily on centralized datasets, feature engineering, and statistical models. But as generative and agentic AI systems grow more capable, the old model is showing its limits. These newer systems do not merely process data: they use data, along with many other elements, to reason, decide, and act. They actively reach into enterprise workflows, APIs, policies, human decision patterns, real-time operational realities, and scattered institutional knowledge.

This led me to a broader question: Is agentic AI still a data capability, or is it becoming something broader as a cross-functional intelligence layer operating at the intersection of Data, Applications, and Operations?

From RPA to AI agents

Almost a decade ago, I experienced the Robotic Process Automation (RPA) wave firsthand. RPA excelled at automating repetitive tasks inside existing applications and workflows. However, it struggled when processes became dynamic, ambiguous, or context-dependent because it lacked adaptive reasoning, semantic understanding, and contextual awareness. RPA could execute workflows efficiently, but it could not truly interpret organizational intent.

Today’s agentic AI systems expose a different challenge. Modern AI agents demonstrate impressive reasoning, language fluency, generative capability, and predictive intelligence driven by advances in machine learning and deep learning. Yet they often lack the operational grounding required for reliable enterprise execution. They may not fully understand workflow state, enterprise policies, escalation paths, operational context, and the institutional knowledge that shapes how organizations actually function.

Taken together, these two technology waves reveal an important shift in enterprise automation. RPA exposed the limitations of automation without intelligence and adaptive reasoning. Modern AI agents have significantly advanced reasoning capabilities, but they are now exposing the limitations of intelligence without sufficient operational grounding. Without that grounding, agentic AI systems can become unreliable, inconsistent, and operationally expensive at enterprise scale.

This is precisely where I believe agentic AI begins to outgrow the traditional “data and AI” organizational model. Building reliable AI systems increasingly requires coordination across applications, operations, governance, security, and enterprise knowledge domains, not just data platforms and model pipelines. Industry discussions on agentic AI operating models are beginning to recognize that autonomous enterprise systems require organizational and operational integration that extends well beyond traditional AI infrastructure concerns.

The fragmented semantic context

Over the years, I have seen enterprise architects meticulously map infrastructure, security, data, services, APIs, and customer channels. Yet what was often missing from these architecture discussions was semantic context: the living organizational knowledge that connects documents, tribal expertise, evolving policies, operational exceptions, and day-to-day business practices.

Recently, I visited a contact center disputes operations unit. A human agent handling a disputes case had access to historical transaction data, customer and merchant insights, real-time workflow state, application APIs, and exception handling logic. Just as importantly, the agent also relied on nuanced policy interpretation, awareness of the latest operational procedures, and precedents from past decisions.

As I analyzed the systems, workflows, and organizational functions involved in making this operation work reliably, an important pattern became clear. Data teams excel at pipelines, lineage, governance, analytics, and transforming raw data into actionable insights. Application teams focus on transactions, codified business rules, APIs, user experience, and delivery velocity. Operations teams prioritize continuity, exception handling, policy adherence, escalation management, and customer outcomes.

My observation is that semantic context is distributed across all three organizational functions. Together, they shape how enterprises actually function. For AI agents to operate consistently and reliably inside enterprise workflows, they must draw contextual understanding from all of them simultaneously. Emerging discussions around context engineering increasingly recognize this challenge of coordinating operational, application, and knowledge-layer context for AI systems. At that point, the challenge is no longer simply about data, applications, or operations individually. It becomes a challenge of building unified semantic context across the enterprise.

Where context already works

One domain already offers a glimpse of what reliable agentic AI interaction could look like: software engineering. Mature and well-maintained Git repositories often work remarkably well with AI coding assistants. Emerging engineering practices around agent-ready repositories, as described by Huseyin Kaplan, are already recognizing the importance of explicit operational structure, contextual documentation, and governed repository semantics for AI-assisted development.

These repositories capture version history, ownership clarity, architectural decisions, review processes, specifications, dependencies, issue discussions, and rich contextual documentation. Over time, they evolve into living, governed knowledge systems that preserve technical intent, operational standards, and architectural governance while creating a unified semantic context around the software lifecycle.

I believe this coherence helps explain why agentic AI systems often perform more reliably in well-structured engineering environments than in many enterprise operational settings, even when those environments contain vastly larger datasets. The differentiator is not merely data volume. It is semantic and operational coherence.

Rethinking agentic AI’s organizational home

All of these observations led me back to my original question: Has agentic AI outgrown the traditional data organization?

As companies move toward increasingly autonomous and agentic systems, the question of where agentic AI structurally belongs is becoming more urgent. Positioning this capability exclusively within the Data organization risks building sophisticated technical capabilities that remain disconnected from real operational workflows. As David Linthicum recently argued in Enterprise AI is missing the business core, many AI initiatives still struggle to integrate deeply with the operational realities of the business. Placing agentic AI solely within Application organizations may similarly lead to fragmented implementations that lack governance consistency, shared knowledge structures, and enterprise-wide operational context.

A more sustainable path may be to treat agentic AI as a genuinely cross-functional enterprise capability, one that deliberately bridges data, applications, operations, governance, and enterprise knowledge domains while drawing unified semantic context from all of them.

This approach does not reduce the importance of strong data platforms, operational rigor, or application engineering. Instead, it acknowledges that true enterprise intelligence emerges from the interaction between these domains rather than from any one domain operating in isolation.

What lies ahead

For decades, enterprises optimized their architecture around storing data, moving data, and processing transactions. Agentic AI introduces a different imperative: enabling systems to reason consistently and reliably within unified semantic context.

The organizations that succeed in the next wave of agentic AI may not necessarily be the ones with the largest datasets or the most advanced models. They may be the ones that create the greatest coherence across their data, operational processes, application systems, governance structures, and enterprise knowledge.

This raises important questions for enterprise leaders and architects. Are AI initiatives still being approached primarily as data-centric programs, or are organizations actively addressing the operational and semantic gaps required for reliable agentic AI? What would a truly governed enterprise knowledge layer look like in your industry, and which organizational functions should be responsible for building and maintaining it?

These are no longer abstract architectural questions. As AI systems become more autonomous and operationally embedded, the answers may increasingly determine which enterprises can deploy AI reliably, govern it effectively, and create lasting business value from it.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?