From data to impact: How the right technology drives generative AI excellence

By Bryan Kirschner, Vice President, Strategy at DataStax

Change management across people, processes, and technologies is a critical part of succeeding with generative AI (genAI). In earlier articles, we’ve covered the human element and how to adapt your processes; here, we’ll take a look at the third: technology.

A recap: A growth mindset and the cognitive value chain

Because deploying technology is a means to an end rather than an end in itself, here’s a recap of the keys to achieving great outcomes by deploying a winning genAI infrastructure and architecture.

With people, the goal is to inspire a growth mindset toward genAI, much as they would take toward any new tool or technique (such as a spreadsheet or the blameless post mortem). But with genAI, they should be pursuing augmentation excellence (“that was a smart way to use it”) and excellent augmentation (“I’m really glad we did that”).

With processes, the goal is to evolve toward a “new normal” way of working in which a cognitive value chain enables knowledge to infuse workflows, at pace and scale, in order to reduce error. It’s conceptually similar to how enterprises developed digital value chains that enabled data to infuse digital experiences, at pace and scale, in order to increase their value.

Our goal here is to point you toward technology that will always help, never stumble, and never stand in the way.

Access to the right data

Let’s start by level-setting on what that entails by using a concrete example that’s likely to become a ubiquitous use of genAI in large enterprises. Here’s what Teresa Heitsenrether, JPMorgan’s chief data and analytics officer, told a Wall Street Journal reporter when asked how genAI will transform work at JPMorgan:

“Think about any place in the bank where people are preparing to go and talk to their clients. Today, you have armies of people running around, pulling briefing memos together and making sure that everybody’s prepped. This is a great way of being able to pull those things together more quickly. We see it in legal, in any place where you’ve got lots of documents, a lot of information to sift through.”

Off the rack, an LLM-powered genAI app such as ChatGPT Enterprise can lend a hand to any user who can craft a prompt and insert documents into its context window. But with important, ongoing workflows such as preparing for customer meetings, sales calls, or contract negotiations, individuals willy-nilly copying-and-pasting from 17 different data sources simply doesn’t make sense.

You want your genAI app developers to be able to build access to the right data sources into tailored enterprise apps, which we represent with the diagram below. The upshot is simple: richer context means better results and greater impact.

Agency and orchestration

But there’s an added twist with genAI. Traditional apps can’t display any agency beyond the data sources and queries hard-coded into them. genAI, on the other hand, can choose to make use of tools and APIs to which its given access.

So the developer tooling layer must incorporate elements of orchestration, too, a concept which we represent with the next diagram below. It’s a matter of bringing not just whatever is in your data estate to bear, but what might be relevant beyond it as well.

For example: if a ticketing database is the system of record for customer support, but one ticket ends with “let’s take this conversation over to Slack,” the genAI app could be equipped to follow the trail. Or if the AI finds conflicting data from internal sources about a customer’s business metrics that are available from a high-quality source such as Dun & Bradstreet, it could tee up the issue and ask permission to make the call.

Finally, for all the human-mind-like behavior genAI can manifest, a genAI app still depends on “math” under the hood to find the most relevant context. And while vector search is table stakes for genAI apps, we know that hybrid search approaches such as combining vector search (for semantic understanding) and lexical search (for exact keyword matching) can improve results.

So what we call a knowledge layer is inserted in order to provide full multi-modal search capabilities beyond the SQL queries that used to be the predominant link between your developers and your data.

The building blocks of AI success

Putting it all together, these three changes – unstructured data becoming a first-class citizen of the data layer; adding orchestration and data access capabilities at the dev tools layer; and the new knowledge layer – will underpin winning processes for leveraging genAI and set up people (both end users and developers) for success with it.

Learn more about DataStax and the technology to help with genAI success.

About Bryan Kirschner:
Bryan is Vice President, Strategy at DataStax. For more than 20 years he has helped large organizations build and execute strategy when they are seeking new ways forward and a future materially different from their past. He specializes in removing fear, uncertainty, and doubt from strategic decision-making through empirical data and market sensing.