UPS delivers customer wins with generative AI

United Parcel Service last year turned to generative AI to help streamline its customer service operations. The in-house developed project, Message Response Automation (MeRA), is already delivering.

MeRA, which was initiated last July and went into beta testing in October, leverages publicly available large language models (LLMs) to automate the handling of some customer issues, providing consistent messaging and significant improvement of agent efficiency, aka handle time, according to the multinational shipping and supply chain management company.

Bala Subramanian, chief digital and technology officer at UPS, sees the company’s foray into generative AI as not only a winner for its customer contact center agents but something to be introduced to other business processes in the near future, he says.

“MeRA’s introduction has been a game changer for UPS, transforming our approach to best-in-class customer service,” Subramanian says. “By alleviating the burden on our human agents, it enables them to focus on more complex and nuanced customer needs.”

MeRA, which has earned UPS a 2024 CIO 100 Award for IT innovation and leadership, automates responses to some of the roughly 52,000 customer e-mails UPS receives each day, according to the company. During pilot testing, UPS earned 50% reduction in the time agents spent resolving e-mails. The LLM gives agents the ability to confirm all responses suggested by the model. This shift to a confirmation role — with standardized workflows — saves the company both time and money, Subramanian says.

Customer service is emerging as one of the top use cases for generative AI in today’s enterprise, says Daniel Saroff, group vice president of consulting and research at IDC.

“Firms that believe their business will be significantly disrupted by genAI within the next 18 months also selected customer-facing applicationsas the top response at 30.8%,” Saroff says, citing research from IDC’s Future Enterprise Resiliency and Spending Survey Wave 1 from January 2024. “Firms are very concerned that without applying genAI to customer-facing apps, they are at high risk of disruption.”

Built to extend

For UPS, contact center use of generative AI is just a springboard. MeRA, which the R&D team first released into limited production in November, will be adapted and extended to all categories of customer contact — and it will be applied to other functions within the enterprise, including human resources, sales, and finance, Subramanian says.

“The journey with MeRA is just beginning,” says the chief digital and technology officer, noting that the tool has prompted UPS to rethink and refine its approach to AI training. “The framework we established is not just a breakthrough for our UPS call center but it’s a blueprint for future AI applications across the enterprise.”

The AI tool dips into the knowledge base used by customer agents to gain access to corporate procedures, as well as data to respond to myriad customer questions. What makes MeRA unique is that it employs a sequential reasoning logic framework dubbed known ‘Chain of Thought’ reasoning, as well as more advanced sentiment analysis capabilities, such as including the ability to customize the tone of the response to customer questions, according to UPS.

Subramanian points to one typical customer request — holding packages for pickup — to illustrate the tool’s sophistication. The request, he says, has at least three possible solutions depending on a variety of factors, such as package tracking history, shipper designated restrictions on packages, and previous delivery attempts by UPS drivers.

“The package tracking history is pulled in real-time from the Internal Visibility portal by an email support agent and shows the history of the package and where it currently is in its journey,” he explains. “Agents must reference this information to know how to respond to various scenarios.”

The model’s use of corporate data such as policies, procedures, and shipping data to automate and enhance customer service demonstrates how generative AI, still in its infancy, is beginning to take steps toward collaboration with humans, analysts note.

“The first half of 2023 saw many use cases focused on how individuals can be more productive in their writing, content creation, or coding tasks, especially across marketing, sales, and development team,” wrote Forrester analysts Rowen Curran and J.P. Gownder, in a blog post last November as UPS was putting its solution into limited production.

“Today’s genAI use cases are moving beyond individual augmentation to reach farther and deeper into the organization to connect organizational knowledge. As these applications access more knowledge, they are interconnecting individuals and teams to enable better collaboration not only between humans but also between humans and machines,” they wrote.

Thought leader George Westerman, senior lecturer at MIT Sloan School of Management and founder of the Global Opportunity Forum, also sees customer service being a great launching point for enterprise genAI strategies.

“One of the most common applications companies are investigating for generative AI is in customer service,” he says. “It can handle routine information gathering and often the first level or two of support. That frees up human agents to handle the more complex questions.”

Although it was conceived and delivered into production in roughly six months, UPS’s phased approach enabled the company to “thoroughly test and fine-tune the system, ensuring that upon its full-scale deployment, MeRA would seamlessly integrate into our operations,” Subramanian says.

Generative AI poses great challenges to CIOs and IT pros today because it is a moving target, he adds. The ever-changing nature of AI tools, as well as the numerous variables involved, presents challenges R&D teams must overcome, he says. Still, Subramanian is confident MeRA can continue to evolve and learn to handle the most complex of customer requests and business processes.

“The generative AI space is constantly changing with new solutions, frameworks, and models being published monthly,” Subramanian says, noting that the team had to teach the model to understand custom e-mails and then follow policies and procedures to deliver the correct response.

UPS noticed in some cases that general-purpose LLMs comprehend or process words and phrases in a manner different from what the company expects for its business. As a result, UPS is investing in training the model on its corporate data set.

For MeRA, UPS started with Microsoft OpenAI LLMs, GPT 3.5 Turbo and GPT4. Going forward, UPS will take a multifaceted approach to its generative AI strategy and will evaluate each framework. “With the technology evolving so rapidly, we are not locked into one model over the other,” according to a company representative.

“We implemented a custom-trained AI agent framework trained on UPS business rules and knowledge base and fine-tuned it to ensure consistent accuracy in providing the right context to the LLM to generate responses,” Subramanian says — an approach he believes will serve MeRA well as the company delivers the tool beyond the contact center, to the enterprise at large.