The next enterprise architecture asset: Ontologies for AI

A data ontology starts with a simple but powerful shift: organizing data by meaning, not just structure.  In practice, it provides a shared semantic framework that defines what data represents, how key entities relate and how that meaning is consistently understood across systems, teams and acquisitions.  By integrating data across silos and domains, an ontology ensures information is interpretable by both humans and machines, enabling a uniform understanding regardless of source or context.  More formally, a data ontology is the explicit specification of concepts, attributes and relationships within a domain, encoded in a machine‑readable form.  This allows systems to reason over data rather than merely store it, transforming disparate tables and records into a cohesive knowledge layer.  

For CIOs, this is not academic, it’s foundational.  A well‑designed data ontology underpins modern data science, enterprise knowledge management and AI, turning raw data into an asset that can be trusted, scaled and operationalized for intelligent decision‑making.

This is important as capturing the semantics (meaning and context) of data, an ontology transforms to a cohesive knowledge base which facilitates informed analysis and inferencing for artificial intelligence and agents.

One of the questions that gets asked is what’s the difference between a semantic model and an ontology?  They both serve different, but complementary, purposes.  The ontology defines meaning and business intent, while a semantic model defines analytical structure and calculation behavior.  The semantic model defines tables, relationships and measures and is the foundation for analytics, typically optimized for BI and reporting.  The ontology is a graph-based, business entity, representing a shared meaning across data, no matter how the data is stored or analyzed. 

An ontology doesn’t replace a semantic model; it stabilizes and standardizes them.  Without ontology, semantic models drift.  Without semantic models, ontology can’t be operationalized in BI.  This matters because an ontology reduces semantic sprawl (i.e., across 100’s of PowerBI datasets), it separates business meaning from implementation, it improves governance, improves trust and reuse, and most importantly, enables AI agents to reason over data correctly.

Ontologies serve as a common vocabulary that maps heterogeneous data sources to a unified model, enabling seamless data integration and improving data consistency and quality.

Perhaps most significantly, artificial intelligence relies on ontologies to supply contextual background that allows AI to reason over data with shared semantics (from complex relationships, drawing logical inferences and communicating across different AI modules).  This ontological grounding allows machines to move beyond keyword matching to true knowledge-driven reasoning, enhancing intelligent decision-making and ensuring consistent interpretation of data across varied applications.

Data ontologies have become fundamental in data science to unify and link heterogeneous datasets, in knowledge management to maintain a consistent conceptual structure for organizational information, and in artificial intelligence to encode knowledge in a machine-readable form.

Why organizations went so long without ontologies, and why that is changing

For decades, enterprises did not ignore ontologies; they simply did not need them.  Traditional enterprise software was built for transaction processing, reporting and automation, not for reasoning.  If systems could store records, execute workflows and produce dashboards, the absence of a shared semantic model was inconvenient but manageable.

This worked because meaning was localized and lived in silos.  Each system encoded its own definition of a “customer,” “policy,” or “contract,” often embedded in application logic, SQL, business rules or in people’s heads.  A CRM system “knew” what a customer was because its developers did.  A finance system “understood” revenue because accountants interpreted it.  If humans were in the loop to resolve ambiguity, enterprises could tolerate inconsistent definitions across systems.  In this case, the institutional knowledge compensated for any semantic drift.

Ontologies have historically been viewed as optional.  Early enterprise ontology initiatives often focused on centralized modeling exercises that struggled to keep pace with business change, and definitions froze while operations evolved.  The result was semantic divergence.  Systems still functioned, dashboards still refreshed, but they no longer meant the same thing across the organization.  The business adapted, but trust slowly eroded.

Another major reason ontologies stalled was traditional analytics and machine learning did not force a correction.  For years, enterprises extracted value from data through point solutions: a dashboard here, a model there, a rule engine somewhere else.  Each use case justified its own data transformation and interpretation.  The cost of reconciling meaning across domains was absorbed by analysts, engineers and operations teams, who manually resolved discrepancies.  Up to this point, AI has delivered value without demanding shared understanding.  Generative AI and autonomous agents fundamentally change this.

When Large Language Models and AI agents are deployed inside enterprises, they operate without continuous human oversight.  They operate autonomously, synthesizing answers, taking actions and reasoning across domains at machine speed.  At that point, ambiguity stops being tolerable.  If “customer” means different things in sales, finance and support, the AI will confidently act on the wrong interpretation or synthesize the wrong answer, which introduces risk, not efficiency.

This is why many enterprise GenAI initiatives stall after impressive demos.  The models are capable, but the environment is semantically unstable.  Enterprises are discovering, often painfully, that they have been outsourcing semantic alignment to humans for decades.  When AI agents act on behalf of those humans, the missing ontology becomes visible.

In the past, ontologies weren’t needed because systems didn’t need to understand; they needed to process.  AI, especially agentic AI, now requires systems that understand the business the way experienced employees do(what exists, how things relate and which rules govern behavior).  Therefore, ontologies are now required to represent a shared understanding of the business for people as well as machines.

The common problems we have been dealing with all this time:

Four main painful areas drive us to use an ontology to solve.  These areas are:

1. Ambiguity: With no common definition, we have terms and data labels that can be interpreted differently by different teams or applications.  Different terms may be used for the same concept, or the same term for different concepts.  This causes confusion and miscommunication, typically requiring manual reconciliation.  This ambiguity makes it hard for AI to align information accurately since a given term may mean different things depending on the context.
2. Poor data integration: Integrating data across silos becomes extremely difficult and labor-intensive.  Each new data source requires a custom mapping to every other data source.  This lack of interoperability means enterprise AI solutions cannot easily combine knowledge from different databases, which limits the scope of analytics.
3. Limited reasoning capabilities: When there are no formal relationships or rules to reason over, AI systems are confined to surface-level pattern matching rather than true understanding.  It cannot infer new knowledge because the domain logic remains implicit.  This renders AI analysis or consistent decision support unachievable without encoding domain semantics.
4. Reduced AI accuracy and reliability: The absence of a semantic framework leads to mismatched or contradictory data interpretations, which decreases the precision of AI outputs. An AI model operating without ontological grounding is prone to errors and inconsistent answers, since it lacks a contextual check on its results.  Enterprises are seeing higher error rates and poor decision outcomes when AI models do not share a common, explicit understanding of the data.

A quick walkthrough of building an ontology

There can be several reasons to create an ontology.  However, we are not just adding another layer and calling that an ontology.  We should take the business context and flow that through the full enterprise AI stack.  We are going to focus on creating this to support all our agentic systems, ensuring that we can confidently build a foundation, the connections and the rules

The foundation layer is the ontology becoming part of the data platform, building a true data asset that is accessible to everyone.  The connection and the rules lead to the semantic contract, which is the grounding for AI agents, telling the agents what actions or interactions are permitted when accessing the data and interacting with the data store.  This becomes the rulebook that makes autonomous AI safe and reliable.  This allows us to build autonomous agents to act safely at scale.

Let’s quickly explore the steps to create the ontology layer.

Step 1: Create the ontology

This foundation layer shouldn’t just be another metadata layer. It should be part of the core and become first-class objects that include entities, relationships and rules.

The entities represent a business concept, for example, an entity of the ontology would be a Store.  You could also have ontology objects for Product, InventoryPosition, Promotion, Sale, etc.  These entities would be bound to physical tables, used as part of a reporting process and in streaming activities (i.e., open/close events).  The goal is to represent all business objects as entities.  While we have a Store, we would also create a business object, for instance, for a Freezer and then we create a semantic connection between the entities that is explicit, queryable and governed. 

So, the Store can have many Freezer entities, but each Freezer belongs to only one Store.  This is important because it allows for questions such as which freezers belong to stores in the North Central region, or which stores are affected if a freezer fails?  We could also have relationships such as Store_has_InventoryPosition or Store_authorized_for_Promotion.  You are using business language instead of database joins.

The rules provide for detection and action.  They can trigger alerts or automation, but more importantly, explanations (i.e., why is this Store at risk?).

This rule uses relationships to automatically enrich the events as it specifies Store, Region and Business Impact.  This is so powerful because it operates on business semantics and not raw telemetry.  The context is inferred through the ontology graph and not hard-coded logic.

Lastly, there are Permitted Actions.  This needs to be a core feature.  What having actions permits is that agents can execute and not just recommend or report.  This is the difference between observing the business and governing what the business is allowed to do.  If the relationship doesn’t exist, the downstream system must refuse execution.  In addition, a very important part of any AI system is how and when to place a human in the loop.  The ontology that you create should have the ability to include, as part of the rules and actions, the ability to outline when it requires human involvement.  And to await explicit authorization or involvement, but also record the decision as semantic fact.

This is why ontologies should be the semantic backbone for agentic AI and real-time intelligence and not just another metadata layer.  The ontology must go beyond metrics and into business state and behavior.

Step 2: Bind the ontology entities to the data structure entities

This step is the semantic integration connecting to the physical data structures.  We map the entity and bind the properties of the business entity to the source properties.  In our Store entity example, we could have properties for a StoreId, a StoreName, a Region, etc.  Each of those properties would be bound to the data properties. After binding each of the Ontology business entities, we would then move on to creating the relationship between the entity types to represent the contextual connections.

Once we have accomplished the full binding, we are ready for our agent interactions.  These agents that interact with the data don’t query the data itself but instead query the ontology.  You aren’t giving the agent schema documentation or example queries.  This is the difference separating what has been done with these new capabilities.  The agent will get the information from the ontology for the entities, their relationship, what’s valid and what’s permitted.  There are no prompt engineering or extensive RAG activities.  The semantic contract is the grounding.

Step 3: Create an agent that interacts with the ontology

As the agent requests data, a semantic query plan is created, which is then translated into the physical execution and finally the response.

If an agent is created to alert or determine if there is “a high value inventory item at risk of being out of stock,” that prompt is converted to the semantic query plan.  The query plan checks to ensure if Inventory has a matching entity, if there is an ontology property called RiskLevel along with a rule for that inventory, checks relationships and ensures that the agent has permission to access the entities and the data required for the request.  The output of the query plan gets converted to a SQL query, obtains the information and then generates a response, not in traditional SQL output but instead, the agent interacts with the business entities in natural language and never interacts directly with the underlying SQL.

An ontology is not a feature — it’s a foundational infrastructure

Ontology is not just another semantic layer but is ultimately the control plane for business meaning in your organization.  By explicitly modeling entities (i.e., stores, inventory, freezers and promotions), binding them to real operational and analytical data, and then attaching rules and permitted actions, the ontology becomes the point where data, policy and execution come together.

Instead of embedding business logic in dashboards, pipelines or application code, an ontology groups it into a shared model that AI agents, automation and humans can all reason over consistently.

The result is a system where decisions are explainable, permissions are enforceable and actions are grounded in live business state.  This allows you to turn data from something we analyze after the fact into something the enterprise can actively operate on at machine speed.

The ontology provides agents with the structure and context they need to act reliably.  You aren’t creating an ontology to decide what is true, but you are setting it up to decide what is allowed so that agents and workflows can operate within guardrails.

To achieve this, we need an ontology in place that allows AI agents to reason without human guidance for every interaction.  The objective is agent reasoning, supported by system validation, rules and governance, with the ability to deliberately insert human‑in‑the‑loop activities where business processes require it.  

Finally, while there are many tools available for creating an ontology, the more important question is where that ontology lives in your architecture.  This decision matters more than the implementation technology itself.  Your ontology should be built into the data layer, where your data already lives, not in the AI layer.  Let me say that again, because it’s critical: your ontology belongs in the data layer, not the AI layer.  When the ontology lives alongside your data, it becomes universally accessible.  Every system, every tool and every AI experience can consume it consistently.  You build it once, and it works everywhere.

The opposite is true when the ontology is embedded in the AI layer.  In that model, it is typically confined to a single tool or a specific set of agents, which forces you to recreate the ontology repeatedly as new tools, platforms or systems are introduced.  This is especially true when they don’t integrate cleanly with the original one.  That path leads to fragmentation and semantic drift.

An ontology is not a feature.  It is foundational infrastructure.  As such, you should be evaluating platforms that treat the ontology as a first‑class citizen of the data layer.  AI requires meaning to be a first‑class concern.  Ontologies are how enterprises institutionalize it.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?