AI agents should retrieve facts, not define them

I have made this mistake before. The goal was to create an executive intelligence agent: an artificial intelligence (AI) engine where C-suite executives could self-serve their analytics and pull trusted numbers for the board.

The promise was strong: personalized dashboards, ad hoc analyses, automated tasks and board decks, all from a single interface. Much faster than the status quo, and without a single new hire.

So, I did the reasonable thing and connected the AI directly to the database. It could query the data, build beautiful charts and refresh dashboards daily.

It was also useless, because the numbers it pulled could not be trusted. Analysts had to step in; Jira ticket counts went up rather than down and adoption never hit the target.

AI agents reach into different systems to perform tasks in the real world. They query databases, call APIs and search documents. They resolve conflicts, interpret ambiguities and report a result. The most valuable agents do all this without human intervention.

That is also where the danger sits. Agents that work unsupervised must answer factual questions accurately. Whether in analytics, sales or customer service, facts are not subjective, and getting them right is where trust lives or dies.

This isn’t a hallucination problem. The agent isn’t fabricating an answer; it is making a decision about data that it never had the authority to make.

Stakes are higher when agents are unsupervised

Human-in-the-loop (HIL) is when an AI agent’s output is verified by a human before it can be dispatched or act in the real world. HIL has become a standard framing for AI applications and a common governance recommendation, particularly in regulated environments.

HIL applications are often internal. They make teams more efficient, reduce cost and increase speed. Coding is a classic example, where engineers review, test and validate code before shipping it.

But there is a class of applications where HIL is not an option, and where trust matters most: a customer service agent answering questions about a customer’s account; an embedded analytics chatbot reporting ARR to a client; an executive intelligence agent putting a number in a board deck. The consequences of getting it wrong are real: a support ticket, a lost customer or a loss of credibility with the board.

In all three, the person receiving the answer cannot check it. And adding a human in the loop would undercut the value of the application in the first place.

A supervised agent is allowed to be wrong because someone verifies the answer, corrects it and is accountable for it. An unsupervised agent is not.

When APIs disagree, your agent makes a choice

We built a four-agent system that could act on behalf of a vendor and answer customer questions in a marketplace setting. The agents needed data about the vendor and customer to provide personalized answers. The system was unsupervised.

But the data came from scattered, siloed APIs. Something as simple as a customer’s address could live in several places, and those systems often disagreed.

The agents had to query multiple APIs and decide which value was correct. The result was unreliable: the same question could produce a different answer across sessions. Personalization was the entire value proposition, and it had become a liability.

What fixed it was resolving the conflicts before the agents ever saw the data. We built business logic that reconciled the raw records and loaded the result into a feature store. The customer’s address, name and budget now had one value each. The agents stopped choosing, and the system became reliable.

That solution required real business decisions: reconciliation rules had to be encoded, and someone had to decide which system won for each field. No amount of prompting would have solved that. Logic that lives in a prompt is subject to interpretation, and there is no guarantee that the same inputs will produce the same outcome twice.

For a production system where trust matters, the answer has to be resolved once and reused, not regenerated on every request.

Choosing the right document before your agent does

The same problem appeared in the unstructured data. Reviews and past customer conversations were stored in a vector database and retrieved through Retrieval-Augmented Generation or RAG. The data was rich and often contained what a customer needed, but multiple sources could disagree.

Sometimes the information was stale: venues move, parking information changes, transportation schedules do too. Other times what was right for one customer wasn’t right for another: instructions for someone arriving by plane are not the instructions for someone arriving by bus, and both are correct.

Again, the agent had to choose.

So, we built a retrieval tool that resolved the conflict before the agent saw the results. We ranked sources by whether the conversation was with the same customer, then the same region, then any customer, with recency weighted throughout.

It wasn’t sophisticated or guaranteed to be perfectly current or correct. It did something more fundamental: it removed unnecessary variability. The same question returned the same answer.

The agent stopped choosing.

Why agents shouldn’t query the database

I ran an experiment to prove to myself I was right. The setup was simple: give LLMs increasingly complex business logic as context and see if they could accurately apply it to data. My hunch was that the more rules there were, the less likely they were to get it right.

The results were worse than I expected. Across nine models, from small to frontier LLMs, the top scores were 97% accuracy at five rules, 73% at fifteen and 43% at fifty. No single model dominated; bigger models did not always perform better, and the average was much lower than the top results.

I was convinced: query generation would never be reliable enough in an unsupervised setting. Context cannot be enforced; it is up for interpretation, and the bigger and more complex it becomes, the less likely the LLM is to follow it correctly.

I needed a different approach. Rather than querying the database directly, the executive intelligence agent would query a semantic layer with clean, unambiguous definitions. The backend engine would then translate the metric query into SQL.

We added a Model Context Protocol (MCP) tool that constrained how the agent could query the metrics: metric names, dimensions, time frame and not much else. By constraining the retrieval, we removed the opportunity for the model to guess beyond what was defined.

This did the trick. Repeated questions about a metric returned the same result, and that result was the one the business had agreed on. The numbers could be trusted. Token consumption and latency were also much lower because answering a question no longer involved SQL generation.

It worked because the semantic layer did the resolution, not the agent. More correctly, a human resolved the metric definition and encoded it into the semantic layer, so the agent didn’t have to guess.

Someone has to decide, and it shouldn’t be the agent

When the data supports multiple answers, but only one of them is valid, the agent is forced to choose. That decision cannot be trusted: the agent does not have the authority to make it, it cannot be held accountable for it, and it cannot be audited afterward because its reasoning leaves no durable trace. Even a perfectly deterministic LLM would produce a consistent answer that no one approved.

The easy answer is to supervise the agent with a human. This adds accountability, but also cost. Humans are slow and expensive, errors still pass through and the resolution logic remains in someone’s head rather than in the system. Worse, accountability often lands with the analyst, sales rep or revenue operator who reviewed the answer, rather than with the person who has the authority to define it.

The more durable answer is a resolution layer between the agent and the data that removes the need for a choice. A feature store reconciles conflicting systems into one value. A retrieval tool ranks sources according to rules someone wrote down. A semantic layer holds metric definitions behind a constrained set of tools the agent cannot route around.

The implementation varies. The principle does not: a human decides once, the system encodes the decision and the agent stops choosing.

This is hard not because of the infrastructure, but because it surfaces decisions someone with authority has to make. What counts as an active customer? Which system wins when two disagree? Which source takes precedence? Those answers belong to the business, not the agent.

There are decisions that can be safely delegated to an agent, others to a human and others that cannot be delegated at all. The skill is knowing which is which.