Scaling gen AI right takes a certain kind of CIO. Are you one of them?

According to McKinsey, only 15% of companies say that gen AI is bringing meaningful bottom-line impact. The more typical experience is to experiment within the classic “tech looking for a solution” trap.

CIOs are on the front lines to turn gen AI’s enormous potential into actual value creation. To do so, they can choose one of three approaches. “Takers” use off-the-shelf, gen AI–powered software from third-party vendors. “Makers” create their own LLMs. And then, in between, are “Shapers,” who integrate outside models into existing systems to create customized uses. For many companies, being a shaper is the most appropriate option, because it’s less expensive and complex than building a foundation model, and more useful than buying off the rack.

So what does it take for Shapers to get out of pilot purgatory and scale up gen AI successfully? These three principles are critical.

Set priorities

In successful tech transformations, actions are closely connected to solving business problems. That’s why CIOs need to work with business unit leaders to set priorities and then make those choices work. The principle is to identify use cases where gen AI advances strategy, which may require shutting down dud pilots and doubling down on those that show promise. Affordability must be part of this analysis as well. Because gen AI is still so new, costs can balloon, making it difficult to scale up. One rule to keep in mind is that for every $1 spent on building gen AI applications, about $3 is needed for change management, including training people and actively tracking performance.

It’s also important to resist the temptation to just cut the techs loose. That can lead to multiple, sometimes overlapping, platforms, which is costly in both money and time. The better approach is to build the infrastructure and applications in a way that provide the flexibility to switch providers or models relatively easily.

Don’t treat gen AI as a tech program

Gen AI is a team sport, and the CIO is the head coach. To have real impact, gen AI has to leave the IT function and be imbedded into the business, which means integrating tech with product, risk, legal, and other departments. One important focus for this cross-functional team is to develop and put in place protocols and standards that support scale. There are different ways to develop such teams, and the CIO will have a large say in their composition and mandate. Some companies have started centers of excellence, while others have chosen to have discrete strategic and delivery units. What matters is that the team collaborates well and knows what it’s trying to achieve, with regular check-ins along the way. The CIO needs to ensure the team acts as builders of value, not just managers of work.

The principle to keep in mind is it’s not about creating different pieces, but making sure they all work together. Each use case needs to be able to access multiple models, vector databases, prompt libraries, and applications. That means companies have to manage a variety of sources, such as applications or databases in the cloud, on-site, with a vendor, or a combination, while ensuring resilience and consistency with existing protocols, including access rights.

Reuse and adapt existing technology

High-performing gen AI solutions aren’t possible without accurate data and an architecture that connects the models to internal data sources. It’s important, for example, to grade the importance of content sources so the model can understand which ones should be given greater weight. Too often, though, companies hunt for perfection, when what they need to do is identify which data matters most and then invest in managing it. Large data programs are the surest path to a slow death. Instead, focus on the data domains that drive value for high priority use cases. Don’t solve for everything. For example, “targeted labeling” can be sufficient to deliver high-quality answers to gen AI queries.

When it comes to code, too, it may not be necessary to reinvent the wheel so much as ensure it’s spinning smoothly. Rather than crafting code for each use case, it’s better to seek solutions that can serve many different ones. Gen AI high performers are almost three times as likely to do just that, according to McKinsey’s latest survey on the state of AI. Reusable code can speed up the development of gen AI use cases by up to 50%. To make this happen, CIOs will need to take the lead in identifying capabilities common across use cases, and then build tools to fit.

Capturing gen AI’s potential has proved to be more difficult than expected. It’s becoming clear that integrating it may require rewiring everything from operating models to technology and data systems. For gen AI to create value as well as excitement, CIOs need to step up and know their place. It’ll be up to them to create the teams, leverage the assets, and guide the strategic thinking that will secure, or hinder, gen AI’s place in the future.