Data mesh: The secret ingredient in enterprise AI success

Conversations about AI technology involve a lot of technical buzzwords and jargon — like large language models (LLMs), agentic AI and reinforcement learning, to name just a few.

Here’s another term that doesn’t always feature as centrally in discussion of AI, but probably should: Data mesh. In many respects, data mesh is the key to unlocking the full value of modern AI in the enterprise.

Now, that may sound like a strange statement, given that data mesh isn’t actually an integral part of AI solutions. You can build AI tools and services without using a data mesh.

However, unlocking the complete benefits that enterprise AI stands to offer is very difficult without a data mesh in the mix. Data mesh may not be a prerequisite for building AI, but it’s an essential complementary technology for building AI solutions that actually create business value.

What is a data mesh?

A data mesh is a type of data architecture that enables decentralized ownership of data. Most data mesh works by connecting the various data sources that an organization owns to make them centrally accessible. However, the underlying data sources remain distinct and can therefore be managed in whichever way is most appropriate on a case-by-case basis.

Data mesh solves the challenge of forcing all of an organization’s data into a single, inflexible location. When businesses adopt that architecture, they end up with a one-size-fits-all approach to data management — which is a problem because in practice, different business domains or departments typically have different data management needs.

With a data mesh, each domain can create data products tailored to its requirements. Data is still centrally accessible and manageable because it’s unified through the data mesh, but it remains flexible enough to support diverse needs and use cases.

The role of data mesh in AI

The main reason why data mesh is critical to effective enterprise AI adoption and deployment is that data is the fuel that powers AI — and the easier it is to access data in a domain-specific way, the more effectively businesses can harness the power of AI technology.

To explain fully what that means, let’s step back a bit and talk about how modern AI technology — especially generative AI and agentic AI, which are the types of AI driving the greatest innovation today — work.

These types of AI solutions are powered by LLMs, which are trained on vast quantities of data. However, a major limitation of commercial LLMs — which are the ones at the core of most enterprise AI products — is that the data they use for training isn’t specific to any business or business domain. It’s generic information.

As a result, the types of commercial LLMs behind the AI tools and services that enterprises are adopting today, such as Microsoft Copilot, lack awareness of the specific requirements or internal operations of the businesses they support. They can write generic code or draft generic emails well. But they can’t draw on business-specific knowledge to do things like develop custom product documentation or understand the state of a particular company’s finances.

Or at least, commercially available LLMs can’t do these things out of the box. It is possible to enhance their capabilities so that they can perform business-specific tasks — but doing so requires exposing the LLMs to data that is specific to the business.

This is where data mesh comes in. When a business has a data mesh in place to organize its data assets, the data mesh serves as a foundation for quickly and efficiently connecting LLMs to the data they need to support advanced use cases.

Benefits of data mesh for enterprise AI

The ability to access data easily for AI model enhancement is just the start of how data mesh helps to unlock the full value of AI technology. The full list of benefits that data mesh provides in the context of enterprise AI includes: 

• Data discovery and availability. With a data mesh, the business doesn’t have to go searching for relevant organization-specific data when it wants to enhance an AI model. The data is already cataloged and available through the data mesh.
• Data quality. By enabling a consistent, centralized approach to data management even when data assets are dispersed across the organization, data mesh encourages high data quality standards.
• Domain-specific data. Because data mesh organizes data based on business domains, it makes it easy to feed LLMs information that is specific to a certain part of the organization or a target use case. For example, if you want an LLM to understand your sales operations, you can point it at the data products owned by the sales team. This is more efficient and more likely to result in better performance than simply training an LLM on a large body of data, only some of which is relevant for specific domains or use cases.
• Up-to-date data. Because data mesh enables decentralized ownership and management of data, it helps to ensure that data remains up to date. This is crucial for effective use of AI because it makes it possible to retrain LLMs whenever data changes — which is important if the models need to be able to act based on the latest information.
• Data security. Data mesh helps to enable high security standards by ensuring that data is accessible only to stakeholders who should be able to view it. In turn, they help mitigate AI security risks, like the possibility that an organization will expose sensitive information to a third-party LLM, and that the LLM will, in turn, leak that data. With a data mesh, access controls can be enforced at the data source, restricting the ability of LLMs to read sensitive information. 

The bottom line

It’s certainly possible to build AI solutions or adopt AI for the enterprise without help from a data mesh. But for organizations that want to use AI to create content or complete tasks that require information specific to a particular business, data mesh is critical. They’re the only effective way of building a data platform that makes it easy and flexible for AI solutions to access the domain-specific data they need, whenever they need it.

Daniel Avancini is the chief data officer and co-founder of Indicium, an AI and data consultancy that helps companies gain an analytical edge through data. He specializes in helping companies build their modern analytics stack using cutting-edge tools and processes for data lake, data warehousing, data governance and advanced analytics.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?