Bank of America’s big bet on AI started small

Bank of America will invest $4 billion in AI and related technology innovations this year, but the financial services giant’s 7-year-old homemade AI agent, Erica, remains a key ROI generator, linchpin for customer and employee experience, and source of great pride today.

Few used the term agent, let alone agentic AI, in 2018, but the bank built a team of software engineers, linguistic specialists, and banking experts to create the small language model, which has been tuned over the years using customer feedback data from the call center.

Hari Gopalkrishnan, head of consumer, business, and wealth management technology at BofA, says the key to Erica’s success and longevity has been its small size.

“We are not writing essays with Erica. We are not trying to write software. We’re trying to understand short bursts of information a customer asks us because they don’t want to hunt and peck on a menu on a screen that’s got 50 different things, and basically be able to translate, what does the customer really mean when they say, ‘I want to pay a bill’?” Gopalkrishnan says. “How do we understand the short burst of conversation that the customer would want?

“We trained the model to do just that,” he says about Erica, which is built on open-source models. “Over time, it went from a model that was 80% accurate to 85% to well in the north of 90% accurate. It also let us be more predictive in what the model would do.” 

Gopalkrishnan says the Bank of America was able to tune Erica during the pandemic to enable customers to apply for PPP loans and handle a multitude of business and consumer needs. He will embrace generative AI and agentic AI offerings as they evolve but believes that most of the bank’s customers requirements can be built in house.

Today, more than 20 million banking clients use the Erica virtual assistant. And more than 90% of the company’s 200,000-plus employee base use Erica for Employees, BofA’s in-house agent for its workforce, resulting in reduced calls to the IT service desk by more than 50%, he says.

Banking on AI

In 2025, Bank of America will use a chunk of its $4 billion AI investments to provide enhanced search and assistance for employees, banking customers, and its Merrill Lynch agent, dubbed Ask Merrill.

The bank is already capitalizing on generative AI applications and pilots that are now beyond the proof-of-concept phase, Gopalkrishnan says. Developers, for instance, are using a AI-based tool to assist with coding and have seen efficiency gains of more than 20%, the company says.

Generative AI is also being used by advisors to prepare for client meetings, saving tens of thousands of hours each year on client engagement and growth. Generative AI is also being used for call center optimization, though the company declined to name the tools being used.

One “internally developed” Bank of America gen AI platform allows global markets sales and trading team to “search, summarize, and synthesize market research and commentary more quickly and efficiently,” the bank stated.

Gopalkrishnan values practical application of agents and existing orchestration technologies to handle consumer and business banking customers over complex agentic AI technologies still in testing, and points to many overhyped technologies, such as metaverse and augmented reality, that have not translated into many business use cases for banking customers.

But Gopalkrishnan sees promise in computer vision and the multimodal capabilities of foundation models, which he is looking into employing to enhance customer satisfaction.

Still, Erica will continue to be the face of customer and employee experience — with more advanced inference and reasoning capabilities added to the back end as requirements surface.

Hybrid cloud fuels innovation

Bank of America spends $13 billion annually on technology — and on partnerships with unnamed consulting firms, rather than going it alone.

Gopalkrishnan, who is the CIO of six of the bank’s eight lines of business, says Bank of America operates a hybrid “hosting strategy” based on a virtual private cloud the bank has operated for years and public clouds as needed. BofA has relationships with Microsoft, AWS, Google, and other clouds, but like many bank CIOs, Gopalkrishnan prefers to keep workloads close for cost and security reasons.

“We have been very effective at scaling it, which lets us get to a point where we’re not paying for bursty volumes,” Gopalkrishnan says, adding it has been “interesting” to see repatriation efforts of some organizations away from cloud computing.

“We’ve always said we’re not going to over-index and over-swing the pendulum,” the CIO says. “Our view is we essentially have a hosting strategy. We’ve got multiple availability zones in our virtual private cloud. We extensively use our virtual private cloud, and as need be, we can burst into public clouds based on the use cases, either for other software providers or for ourselves.”

Bank of America also continues to bet big on the mainframe, which has helped it periods of sharp volatility in the stock market of late.

“The mainframe continues to be a very important strategic platform. But over time, we’ve absolutely modernized, figured out what workloads actually belong better in a distributed environment, which workload should be more horizontally scalable across multiple availability zones, and which workloads would be irresponsible for us to just go through a lot of money and rewrite just for the sake of rewriting,” he says.

Data aggregation and data cleansing have also been in the playbook as Bank of America continues its foray into analytics and AI, and Hadoop and Snowflake are some of the data platforms in use, he hints.

“We’ve been modernizing our data plan,”  Gopalkrishnan says. “We have a pretty extensive body of work around data analytics, sourcing the data from the right place, making sure it’s clean, making sure it’s well governed, making sure it’s dealt with in a responsible way.”

He adds: “Everything we do in AI goes through a governance process that has 16 different pillars [such as] bias and transparency.”

Getting into the gen AI game

As the company digs further into generative AI, practicality will be the core objective of technology selection — although Gopalkrishnan acknowledges BofA is likely to use more advanced foundation models. But this financial services company will explore the simplest solution to produce outcomes and will not be tied to any one vendor.

“Our goal is to be model-agnostic, because things change in the industry significantly with time. Reasoning comes along, token pricing changes, new innovations come along,” he says. “We don’t want to be wedded to any given model. Essentially, we look at a use case, we look at data classification, we look at our capabilities, and then put together what the right solution for the problem.”

Gopalkrishnan claims the training, inference, and reasoning innovations of foundational models are great but he will continue to use off-the-shelf recipes and established solutions to address the evolving digital needs of his customers and employees. He is not looking for a rip-and-replace platform.

“We’re not looking to chase the next shiny thing that just got announced somewhere because there’s plenty of things that can be done with what’s already available through simple common sense of AI agents with basic orchestration,” he says.

Forrester analyst Brian Hopkins calls BofA’s technology approach “pragmatic precision” and point outs Erica, which has handled more than 2.4 billion interactions with a 98% containment rate, as a masterclass in scaling digital engagement without compromising trust.

“They’ve taken a more cautious path into gen AI but I’d argue it might turn out to be the smart play,” he says of America’s second-largest bank. “Trust is the currency of banking, and gen AI still carries real risks [such as] hallucinations, lack of explainability, and security gaps. Why risk trust when what [BofA] built already works and works well? With theirscale, a misstep could be costly. This isn’t a knowledge business like consulting, where gen AI has had a more immediate impact.”

Now that it’s entering the gen AI phase, the bank is doing it from a position of strength — clean data, clear business goals, and deep experience operationalizing AI, the analyst adds. Their invest-once, reuse-many-times model could pay off meaningfully over time.

“In the end, while they’re not flashy, I found them quietly effective,” Hopskins says. “I wouldn’t be surprised if they outpace the competition once the gen AI dust settles.”