Oracle delivers semantic search without LLMs

Oracle says its new Trusted Answer Search can deliver reliable results at scale in the enterprise by scouring a governed set of approved documents using vector search instead of large language models (LLMs) and retrieval-augmented generation (RAG).

Available for download or accessible through APIs, it works by having enterprises define a curated “search space” of approved reports, documents, or application endpoints paired with metadata, and then using vector-based similarity to match a user’s natural language query to the most relevant of pre-approved target, said Tirthankar Lahiri, SVP of mission-critical data and AI engines at Oracle.

Instead of retrieving raw text and generating a response, as is typical in RAG systems that rely on LLMs, Trusted Answer Search’s underlying system deterministically maps the query to a specific “match document,” extracts any required parameters, and returns a structured, verifiable outcome such as a report, URL, or action, Lahiri said.

A feedback loop enables users to flag incorrect matches and specify the expected result.

Lahiri sees a growing enterprise need for more deterministic natural language query systems that eliminate inconsistent responses and provide auditability for compliance purposes.

Independent consultant David Linthicum agreed about the potential market for Trusted Answer Search.

“The buyer is any enterprise that values predictability over creativity and wants to lower operational risk, especially in regulated industries, such as finance and healthcare,” he said.

Trade-offs

That said, the approach comes with trade-offs that CIOs need to consider, according to Robert Kramer, managing partner at KramerERP. While Trusted Answer Search can reduce inference costs by avoiding heavy LLM usage, it shifts spending toward data curation, governance, and ongoing maintenance, he said.

Linthicum, too, sees enterprises adopting the technology having to spend on document curation, taxonomy design, approvals, change management, and ongoing tuning.

Scott Bickley, advisory fellow at Info-Tech Research Group, warned of the challenges of keeping curated data current.

“As the source data scales upwards to include externally sourced content such as regulatory updates or supplier certifications or market updates that are updated more frequently and where the documents may number in the many thousands, the risk increases,” he said.

“The issue comes down to the ability to provide precise answers across a massive data set, especially where documents may contradict one another across versions or when similar language appears different in regulatory contexts. The risk of being served up results that are plausible but wrong goes up,” Bickley added.

Oracle’s Lahiri, however, said some of these concerns may be mitigated by how Trusted Answer Search retrieves content.

Rather than relying solely on large volumes of static, curated documents that require constant updating, the system can treat “trusted documents” as parameterized URLs that pull in dynamically rendered content from underlying systems, according to Lahiri.

Live data sources

This enables it to generate answers from live data sources such as enterprise applications, APIs, or regularly updated web endpoints, reducing dependence on manually maintained document repositories, he said.

Linthicum was not fully convinced by Lahiri’s argument, agreeing only that Oracle’s approach could help reduce content churn.

“In fast-moving domains, keeping descriptions, synonyms, and mappings current still needs disciplined owners, approvals, and feedback review. It can scale to thousands of targets, but semantic overlap raises maintenance complexity,” he said.

Trusted Answer Search puts Oracle in contention with offerings from rival hyperscalers. Products such as Amazon Kendra, Azure AI Search, Vertex AI Search, and IBM Watson Discovery already support semantic search over enterprise data, often combined with access controls and hybrid retrieval techniques.

One key distinction, between these offerings and Oracle’s, according to Ashish Chaturvedi, leader of executive research at HFS Research, is that the rival products typically layer generative AI capabilities on top to produce answers.

Enterprises can evaluate Trusted Answer Search by downloading a package that includes components such as vector search, an embedding model to process user queries, and APIs for integration into existing applications and user interfaces. They can also run it through APIs or built-in GUI applications, which are included in the package as two APEX-based applications, an administrator interface for managing the system and a portal for end users.

This article first appeared on InfoWorld.