The great AI reset: CIOs pivot from pilots to business value

AI experimentation is fertile ground at TIAA with dozens, if not hundreds, of concepts and pilots taking root at any given time. Yet when it comes to enterprise scale, the 107-year-old financial services company has prioritized only a half-dozen AI deployments, most tackling important but unflashy use cases.

The approach has nothing to do with lackluster interest in AI. Rather, it’s a measured deployment strategy that TIAA technology lead Sastry Durvasula says is calibrated for long-term results.

“Right now, there’s a lot of sizzle and less steak,” says Durvasula, chief operating, information, and digital officer at TIAA, which specializes in retirement. “Everyone wants to show the next-generation, interesting use cases, but when it comes to AI at scale, you’ve got to solve for the core, what I call, boring problems — not those that are tangential to the business.”

Durvasula’s intentional approach to scaling AI comes at a critical inflection point in the arc of AI adoption. After the initial frenzy over generative AI, and more recently agentic AI, companies are facing a reckoning of sorts, as they acknowledge that the initial surge of use cases worked wonders for individual productivity, but had minimal impact on business transformation — and more importantly, the bottom line.

According to a report from MIT’s NANDA initiative, US companies have poured between $35 billion and $40 billion into gen AI projects, yet only about 5% of those efforts have yielded substantive revenue growth. The vast majority, the report found, deliver little to no measurable impact on P&L, not because of the quality or performance of AI models. The challenge is targeting the right use cases, and according to the MIT report, overcoming technical issues, specifically enterprise integration as well as setting up models to learn and align with corporate workflows.

Companies are also performing a reset on the hyper experimentation phase of AI due to healthy skepticism about AI model hallucinations and bias. They’ve also found many users to be reticent about embracing a technology they fear may ultimately take their jobs.

“We don’t have the trust aspects fully built out and we haven’t gotten to the optimal levels of performance where we can truly measure AI’s value,” notes Rani Radhakrishnan, a partner for consulting solutions at PwC. “What it is going to take is to go through the full cycle. It’s a matter of time.”

Back to the basics

Moreover, IT leaders are also seeing wisdom in getting back to the nuts and bolts of new technology deployments, including establishing frameworks and methodologies as opposed to chasing a fast-moving technology on a whim. Compared to past technology waves, AI is accelerating more quickly and there are a lot more unknowns.

“In the past, there were frameworks and methodologies for doing software engineering that you could follow and execute well,” says Kenneth Spangler, retired executive vice president and CIO of global operations and technology at FedEx and currently a strategy and technology executive consultant. “There’s no real framework for this yet, and that’s a problem. Things are still evolving.”

Enterprise impact and adoption will come when companies trade up unbridled experimentation with implementation practices that stand the test of time, even as they are modified for the unique characteristics of AI. Companies like TIAA are starting to follow that path, building out an enterprise AI deployment framework that encompasses formalized governance structures, metrics, change management initiatives, and training and literacy programs designed to parlay individual use into an end-to-end strategy that advances enterprise business goals.

Governance has been a major part of Regeneron Pharmaceuticals’ pivot from experimentation to AI at scale. The company has devoted a lot of time to examining the risks it is — and isn’t — willing to take, recruiting help from outside experts that understand the industry and business model, as well as its data and compliance requirements, according to Bob McCowan, senior vice president and CIO for the firm.

That AI governance work also needs to establish the proper controls to manage and enforce the governance practices — for example, building a mechanism that would allow or disallow data to be fed into an LLM when doing classification work on documents, he says.

To complete that work, Regeneron revisited data access and data management controls, especially to address historical differences in various areas of the business. Regeneron established a governance team with representation from different business groups to debate the issues and hammer out a structure. McCowan sees the governance work as central to the company’s ability to launch a handful of gen AI applications at scale, including its own version of a copilot that enhances employee productivity without any of data being shared outside of the company’s four walls.

“Governance, done correctly, is the brakes that won’t slow you down, but allow you to go faster,” McCowan says. 

Tractor Supply Co. has also put a lot of muscle into AI governance, including the implementation of the right security controls, data quality guardrails, and ensuring the quality of prompts remain strong. But even before that effort, the company’s approach to AI started with identifying use cases with the highest potential to solve business problems, not just boost individual productivity.

“We don’t lead with the technology — we lead with the problem,” explains Rob Mills, executive vice president and chief technology, digital, and strategy officer for the rural lifestyle retailer. “We start with the organization’s pain points and let AI earn its way into the solution.”

Another critical aspect of Tractor Supply’s roadmap for scaling AI is LLM standardization — in this case, collaborating in close partnership with OpenAI and ChatGPT. Tractor Supply has integrated OpenAI’s platform into enterprise workflows and included hooks that tune LLMs with company data to serve specific personas, such as team members, or use cases, such as customer training.

One such LLM is designed to help team members on the shop floor better assist customers by serving up pertinent data in real-time. Say a customer has a flock of chickens and wants to produce more eggs — the team member–specific LLM equips the on-the-floor sales member with the right information to advise the customer about relevant products and what needs to be done. The use case quickly went from a one- to two-store pilot to deployment at scale with 200,000 to 300,000 questions added monthly. Even better, there has been a direct correlation to increased sales, Mills says.

“By standardizing our platform, we can train once, govern once, and scale everywhere,” Mills says. “That discipline keeps costs in check and turns insights into impact.”

That disciplined focus to scaling AI by no means diminishes Tractor Supply’s appetite for experimentation. Team members are provided with tools, training, and a secure environment to explore and experiment with AI to boost their own individual productivity and automation. The more successful use cases are often served up to the broader team member community once they are proven out.

“We have built a culture of safe experimentation,” Mills says. “Teams try ideas inside clear guardrails and when something shows real value, we scale it. That’s how we turn thousands of prompts and prototypes into consistent business outcomes.”

Metrics and structures: Two keys to success

As part of its AI-at-scale roadmap, Webster Bank has developed a three-legged stool strategy built around a formal governance structure, an AI business users group, and AI technology build and design practices.

The AI Governance Committee, which formally reports to the Technology Committee of the Board, has representation from IT, data, cyber, privacy, compliance, legal, and audit teams, tasked with evaluating the scalability of each pilot to align with the bank’s risk profile and ensure there’s no model bias. The AI business user group, which includes people from every line of business and corporate function, serves as champions and evangelists for AI. The committee brings compelling use cases to the table, which are then formally evaluated to gauge their potential for enterprise value and success.

“This is the grassroots piece of the stool,” says Vikram Nafde, executive vice president and CIO for the bank. “A good portion of the ideas come from operations and technology — that’s where we’ve seen the biggest lift in terms of AI bringing business value.”

To date, the grassroots organization has collected 150 potential AI use cases with about a dozen in production as a result of the process, Nafde says. The team also works on developing the right metrics for AI implementations — not after the fact, but before the first line of code is written.

“We identify KPIs early and test them throughout the process,” Nafde says. “If you can’t measure, you don’t know if something is working or if it should even scale.”

The third leg of the stool involves bringing selected use cases to life. A dedicated agile team called Quantum Minds works to design and build these AI solutions under the leadership of the head of AI who reports to Nafde. Webster also partners with hyperscalers and other technology solution providers to accelerate the velocity of use case buildouts.

At TIAA, every potential AI initiative is evaluated against an ROI framework to consider the impact on topline growth, bottom line improvement, client experience, or risk. Any project earmarked for enterprise scale has metrics designed into the plan to ensure accountability. That goes for the current roster that includes MYGAIT (My Generative Agentic Intelligent Technology), an AI assistant deployed to thousands of associates globally, and an empathy agent, which helps call center employees better engage with customer complaints in a more personalized, empathetic way.

“2026 is the year where we are baking these things right into the business plan to make sure the ROI promise is actually paying off after implementation,” Durvasula says. “It’s the best way for accountability — if you don’t hit the metrics, then you miss the plan.”

Another majority priority in the TIAA roadmap for scaling AI is change management.

Working closely with his C-suite partners, including the CFO and chief people officer, Durvasula and his counterparts are addressing the big issues, from figuring out what the workforce of the future looks like with AI in the mix to designing the right risk and governance controls and establishing metrics to ensure ROI.

“Change management in an AI-powered organization has to be managed carefully,” he says. “There’s a lot of anxiety about losing jobs to AI so there’s a level of optimism you have to portray to help the organization get to the next stage.”