Ally Financial finds gen AI success with 3 guiding principles

Bank holding company Ally Financial is determined to stay at the cutting edge of technology in the financial industry. Today, that means leveraging gen AI to transform its business. But Ally is part of a highly regulated industry, which has seen many banks and financial institutions delayed by regulations. Ally’s answer? Develop an AI platform and write a gen AI playbook to allow it to move quickly without shortchanging on security and governance measures.

“What I’ve learned is disciplined, intentional tech investment and tech capabilities are critical to take advantage of tech evolution,” says Sathish Muthukrishnan, Ally’s chief information, data, and digital officer.

Muthukrishnan notes that after seeing the rise of LLMs and gen AI in 2022, it was clear the company needed to shift its AI ambitions to make use of the new technology. The question was how to incorporate it without sacrificing security.

To get company leadership onboard, he says it was essential to prove to them that Ally could use gen AI responsibly.

“What I was very clear on from day one was that while you might be seen as an industry leader and get recognition for adopting gen AI early, even a small misstep would put us back 100 steps from the half step we have taken forward,” he says. “We didn’t want to be in that position, so we needed to protect ourselves.”

Muthukrishnan and his team developed three simple guiding principles: Experiment with intern-facing use cases first, always have a human in the middle, and never expose any Ally data externally to LLMs.

“There was no roadmap, so we built one,” Muthukrishnan says. “A secure, reliable and scalable platform from which to run all AI applications. We wanted to show that AI could be done responsibly, while protecting customer privacy and securing company data, and we could effectively train our employees to use the technology.”

To execute on those principles, Ally built and launched Ally.ai in June 2023, a proprietary platform for all the company’s AI applications, and it earned Ally a 2024 CIO 100 Award for IT leadership and innovation. It encompasses traditional ML and MLOps capabilities along with gen AI capabilities. Ally’s in-house development team built and runs the platform with the support of Ally’s cloud via AWS and Microsoft Azure OpenAI, though Muthukrishnan notes the platform is LLM-neutral.

“All.ai came to the rescue because it had the ability and controls to effectively and safely use all these large language models,” he says. “That’s why we were also not afraid to go all-in on OpenAI, because Ally.ai has the ability to talk to OpenAI today, but tomorrow it could talk to Claude, Llama, or it could talk to any other new LLM.

Keeping gen AI in check

Despite internal excitement around the platform, Muthukrishnan says AI governance remained top-of-mind. The company adapted its Ally Technology Operating Model (ATOM) for gen AI. The model has five core pillars that guide the introduction of all new technology and provide a roadmap for assessments, evaluations, and approvals: Discover, Ideate, Elaborate, Execute, and Measure.

The company also established an AI Working Group to work in conjunction with its existing Governance Group. Together, they formed an internal team of professionals in financial service fields including regulatory compliance, risk management, and audit, among others. The team reviews and advises on gen AI use cases.

To help guide the team on implementation of both classic AI and gen AI, the AI Working Group also developed the Ally AI Playbook to empower Ally’s business lines to explore AI use cases, plan for pilot programs, and move them into production in a responsible, considered way. Muthukrishnan says the playbook creates a common language of understanding across the company.

“You can come at [a project] from any language, but the AI Playbook is the common translator to tell you, ‘This is what you’re asking, and this is what you have to go and understand,’” he says.

In addition, the company created a basic AI training course for all 11,500 employees and instituted AI Days, where external industry leaders and internal speakers share their struggles and successes. There are also regular demonstrations of how Ally is using gen AI internally.

“All of this has created a change evolution where everybody is participating,” Muthukrishnan says. “Most of these AI Days are people outside of technology. We have an average of 1,200 people participating for over four hours every six to eight weeks. We feel the change momentum and the transformation that’s happening, and everybody across Ally participating.”

What’s under the hood

Ally built Ally.ai on its dedicated cloud infrastructure with its own private network, creating a fully secured environment with no external access. The platform removes Personally Identifiably Information (PII) before the LLM is engaged, and other Ally data stays private and isn’t shared. For additional security, it uses tokenization before data is sent to the private LLM on Ally.ai. The team stays in regular communication with regulators and collaborates with third-party LLMs like Azure OpenAI and AWS to protect against data leakage, and from the foundational models learning from Ally data.

Soon after launching the platform, Ally introduced two gen AI use cases: call summarization and marketing.

The call summarization capability was developed to support Ally’s Customer Care and Experience Group and leverages Microsoft Azure and Azure OpenAI Service to produce detailed documentation for each call taken by its call center associates. It provides real-time access to summaries of tens of thousands of customer service calls per week.

“We do about 10,000 calls on average per day,” Muthukrishnan says. “It lifted the cognitive load for the customer care associates to singularly focus on our customers rather than keeping track of the conversation and trying to create a summary at the end of the call.”

The marketing team capability, on the other hand, uses Ally.ai’s LLM chat and prompt functionality to help marketers produce creative campaigns. Ally says it’s helped reduce the time needed to produce creative campaigns and content by up to three weeks, primarily with early-stage tasks like research, first drafts, and naming exercises, resulting in an average time savings of 34% compared to typical processes without AI.

Muthukrishnan says the key to success with both capabilities was ensuring participation from all corners of the company.

“This is an enterprise transformation not just a tech transformation, and we had to be patient, not just in advancing the use of the technology but in how and who used it and who drove that change,” he says. “We had one person we hired externally, intentionally to drive AI models, but the rest of the organization was all internal employees.”

Casting a wider net

The bank is working on a number of additional gen AI uses cases. For instance, its audit team is using Ally.ai to create risk audit control metrics that assign risk factors to every capability they audit. That task has traditionally taken several days, but gen AI allows them to complete the task in hours.

Muthukrishnan says he feels success with gen AI, and any cutting-edge technology comes down to three things:

1. Focused, intent-filled technology investment and build.
2. The courage to experiment with new technologies, but with an intent to showcase value so as to bring the entire organization together.
3. The patience and thoughtfulness to educate the entire company.

“Don’t just experiment and invent within the four walls of tech,” he says. “Showcase the value of tech [to the entire company], speak their language, and bring them along by educating and empowering the entire organization.”