The next AI advantage isn’t a bigger model — it’s a better semantic layer

Every AI roadmap review I sit through this year eventually lands on the same question: which technology stack/model should we use? It’s the wrong first question to start with. Gartner projects that by 2027, organizations will deploy small, task-specific AI models at roughly three times the volume of general-purpose large language models. In my experience running agentic pilots across a large enterprise, those specialized models succeed or fail almost entirely on something that has nothing to do with which model you picked: whether your own business actually agrees on what its data means.

That’s a harder problem than it sounds, and it’s not the same problem as data quality. My team has spent years measuring data quality the traditional way — is the field populated, is the number accurate, is the record deduplicated. Those checks matter, but they answer a narrower question than the one agentic AI actually depends on. IBM’s own explainer on the topic puts it well: a semantic layer is the piece of enterprise architecture that translates complex, technical data structures into the business terms people actually use day to day. Clean data can still be semantically ambiguous. A field can be perfectly accurate and still mean three different things depending on which team is looking at it.

Why accurate and understood are not the same thing

I recently worked with a large MedTech firm where I led data and governance strategy to build AI agents; the term “active customer” existed in three separate systems, defined three separate ways: one team counted anyone with an account open in the last 90 days, another counted anyone with a transaction in that window, and a third counted anyone who hadn’t formally churned. Every one of those definitions was technically accurate — the underlying data was clean, well-governed and correctly populated. None of them agreed with each other.

For years, that ambiguity lived quietly inside dashboards, where a human analyst could squint at a number, ask a colleague which definition it used and adjust their interpretation accordingly. Agentic AI removes that human buffer. When an agent pulls “active customer” to decide who gets a retention offer or who gets flagged as a churn risk, it doesn’t know there are three competing definitions sitting behind that phrase — it picks whichever one the underlying table happens to expose and acts on it with full confidence. The output looks decisive. It’s just decisively wrong for two-thirds of the organization.

What happens when agents inherit your organization’s ambiguity

I learned this the expensive way. When we ran an early agentic pilot on a customer retention workflow last year, the agent flagged a segment of accounts as “at-risk” that the finance team didn’t recognize as at-risk at all. It wasn’t a model problem — the reasoning was sound given the data it was pointed at. It was a “definitions” problem: sales, finance and customer support each had their own notion of “at-risk,” built into three different systems at three different points in the company’s history, and nobody had ever forced them to reconcile. We ended up spending more time aligning those definitions across departments than we spent building the agent itself.

That experience isn’t an outlier. Informatica’s 2026 survey of 600 global data leaders found that half cite data quality as their top barrier to deploying agentic AI, and 57% call data reliability the wall standing between a pilot and real production use. In my own postmortems, a meaningful share of what gets logged as a “data quality” issue in surveys like that is really a semantic issue wearing a data-quality costume: the data was accurate; it just didn’t mean what the agent — or the team reviewing its output — assumed it meant.

This matters more for agentic systems than it ever did for reporting, because agents don’t just surface an ambiguous number for a human to sanity-check. They act on it. A dashboard with a wrong “at-risk” count is an annoyance. An agent that auto-triggers retention offers, or worse, credit decisions, based on the wrong definition of “at-risk” is a business risk with a dollar figure attached.

The same pattern shows up anywhere a term carries more weight than its plain-English simplicity suggests. “Revenue” means something different to a sales team tracking bookings than it does to finance tracking recognized revenue under accrual accounting — a distinction that rarely causes trouble in a quarterly business review, where a human in the room can ask which number is being quoted. Hand that same ambiguity to a pricing or forecasting agent, and it will pick one definition, apply it consistently and confidently, and generate a forecast that’s internally coherent and organizationally wrong. The failure mode isn’t a crash or an error message. It’s a plausible-looking number that nobody thinks to question until it’s already driven a decision.

Build the semantic layer before you scale the next model

The fix isn’t a bigger model, and it isn’t even primarily a bigger data quality initiative, though that’s the instinct most organizations reach for first. It’s a deliberate semantic layer — the shared, machine-readable set of business definitions that sits between your raw, clean data and whatever model or agent is querying it. In practice, that means going through a sequence most data teams skip because it’s slower and less exciting than deploying the next pilot.

Start by auditing and cataloging the terms most central to your highest-priority use case — not every term in the business, just the three or four that the agent will actually touch. Force the departments that use those terms to sit in a room and agree on one definition, even when that conversation is uncomfortable and political, because it usually is. Then encode that agreed definition somewhere machine-readable and enforced — not just written down in a glossary document nobody opens again after the kickoff meeting — so every model, dashboard and agent that touches that term resolves it the same way.

None of this needs to wait for a multi-year data governance overhaul, and I’d actively discourage treating it that way. The mistake I made early on was assuming semantic work had to be comprehensive to be useful — mapping every term across every system before touching a single agent. It doesn’t. Scoping it tightly to one workflow, getting the two or three business functions that actually touch that workflow to agree, and encoding that narrow agreement is a project measured in weeks, not quarters. The comprehensive enterprise glossary can, and probably should, come later, built one validated use case at a time rather than designed top-down in a conference room before anyone has tested it against a real agent.

CDOs and CIOs would have to ensure that this is also a mandatory shift for data teams themselves, not just a technical build. The job used to be building reports and charts for humans to interpret with their own judgment layered on top. It’s increasingly about building and maintaining the context that lets a machine interpret data correctly without a human in the loop to catch the ambiguity. That’s a different skill set than most data teams were hired for, and it’s worth saying so explicitly when planning headcount and training for the next year, rather than assuming the existing analytics team will absorb it by osmosis.

The organizations that come out ahead in the next phase of agentic AI won’t necessarily have access to a better model than their competitors — most of us will be drawing from a similar, increasingly commoditized set of models within the next year or two. What they’ll have is a semantic foundation clear enough that a smaller, specialized model can act on it with confidence, and a team that already went through the unglamorous work of agreeing, in writing, on what the business actually means by its own words. That’s a much harder thing for a competitor to copy than a model subscription — and it’s the argument I now make first in every AI budget conversation, well before anyone asks me which model we’re using.