The agentic AI transition is underway

November 2022 was a big milestone for AI when ChatGPT was released. But in terms of enterprise transformation, November 2024, when Anthropic released MCP as an open standard, was maybe an even bigger transition point.

MCP ushered in a new age. Before, AI was primarily used to answer questions, but with MCP, AI models could start directly interacting with data sources and tools, allowing them to carry out tasks on behalf of users, turning AI chatbots into AI agents. Most leading-edge AI platforms today are agentic in nature, doing work like sending emails, making purchases, and performing scheduled tasks. But that’s just baseline functionality.

When integrated with corporate workflows, AI agents have the potential to carry out business processes that were previously either too complex, unpredictable, or costly to automate. And deploying agentic AI in the enterprise could be as simple as allowing AI assistants to access email systems and document repositories, or enabling pre-built agentic functionality in Salesforce and other enterprise software platforms.

Earlier this year, Deloitte released a survey of more than 3,200 executives showing that 75% of organizations use at least some agentic AI. And within the next two years, 95% said they expect to use AI agents.

But to gain competitive advantage, enterprises need to do more than just deploy the same AI agents everyone else is, and some companies are doing just that. In the Deloitte survey, 30% of enterprises are already redesigning key processes around AI while 34% are even starting to use it to deeply transform their business.

Today, 20% say they’re using agentic AI moderately, 2% extensively, and 1% say it’s fully integrated as a key component of their operations. Yet within two years, 46% expect to be using agents moderately, 23% extensively, and 5% as key components of their business.

AI is already paying off for some

AI is showing signs of transformative impact in terms of increased efficiency and productivity, more than double from a year ago, according to Deloitte’s findings. Some leaders are even seeing an impact on their bottom line, with 20% of enterprises surveyed reporting increased revenue. Here’s how AT&T is doing it, for instance.

“Last year we had five-X cash ROI on our AI investment,” says Andy Markus, the company’s chief data and AI officer. “We’ve hit the flywheel.”

AT&T has built its own AI platform, he says, and that investment is paying off. The platform is model-agnostic, he adds, and the company controls the knowledge it uses, grades its accuracy, and monitors the platform for governance and accountability.

“It doesn’t require as much development anymore,” he says, “and now we’re just generating more value by applying this core best-in-class functionality.”

AT&T didn’t bet on any one particular AI vendor either since the technology is changing so rapidly. “We can’t be beholden to any one provider,” he says. “We can change out the model and framework. What makes it useful is our data, knowledge, and expertise. That makes the difference.”

Another thing that helps drive ROI is that the platform is accurate and reliable. “When we apply it to our own data, we’re now converging on 100% accuracy for complex queries,” he says. “That’s what drives value.”

AT&T isn’t new to AI, Markus says. “We have a deep history of AI, including machine learning and classical,” he says. “But we’ve ramped it up with generative AI and agentic AI.” Today, the company has more than 1,000 agentic workflows either being built or already in production, he says, yet it’s trying to minimize the total number of agents by building them to be multi-purpose. “We’re making these agents super flexible and bringing the right information to them and dictating the direction they take.”

Agentic AI might feel like it’s a new thing, he says, but at AT&T, it’s matured to the point where it’s starting to feel like traditional software. “You build and optimize, and we’re in that cycle,” he says. “We do that in a routine fashion.”

So AT&T is ahead in the game, and that’s the strategy. While many companies are waiting for AI platforms to mature to the point where deployment is easier, more reliable, and more cost effective, AT&T isn’t, and just builds what it needs.

“By the time a lot of it becomes commonplace, we’re already there,” Markus says. “We’ve already figured out the problems at the leading edge and are ready to make this standard practice.”

Maximizing the resources within reach

Traditionally, the insurance industry has been slower to adopt AI than others due to legacy systems, regulations, and, of course, because whole business model is about avoiding risk. But some companies, like 180-year-old New York Life, are bucking the trend by modernizing data infrastructure and rethinking business processes and workflows.

Matt Marze, CIO at New York Life Group Benefits Solutions, says his group has been on a technological transformation journey for over a decade. “And now we’re supercharging it,” he adds. “And both generative and agentic AI are part of that strategy.”

His part of the company focuses on the group benefits market, offering employer-sponsored life and disability insurance, as well as leave management and wellness products. And two core aspects of the transformation are the overall modernization of the group’s technology and building its AI platform.

“We’ve done a lot of tech modernization to drive our cloud adoption and strategic data management, and we’ve consolidated all our business applications on a set of platforms, both commercial and our own,” says Marze. “We’ve also assembled an AI operating platform or system to give us full-stack AI capability.”

Similar to what AT&T is doing, New York Life Group Benefits is building its own custom AI business engine.

“It’s a lot of cloud-native capabilities in AWS and other clouds, and open source tools from companies like LangChain,” he says. “It’s not point solutions. We’re steering clear of embedded AI from commercial platforms from Salesforce or other providers. We’re looking to enable our own AI stack and drive it across our business.”

The reason some companies have to build their own AI infrastructure is because the vendor landscape in this space is still lagging, says Kevin Martelli, consulting AI solution development leader at EY Americas. “There’s not something bringing all these pieces together to make it easy for the business to operate,” he says.

Instead, companies often pull together what they need from their databases and other tools. Some functionality is provided by the foundational model companies themselves, he says. “But a lot of times it’s up to you.”

At New York Life Group Benefits Solutions, initial priorities included the service operation, employee productivity, and sales growth.

“We have both generative AI and agentic solutions deployed today,” Marze says.

Take compliance, for example. “We voluntarily go through SOC 1 and SOC 2 audits,” he says. “It’s traditionally a very labor-intensive process but we’ve deployed agentic AI to gather a lot of the evidence we need to collect and help prepare it for the folks who drive the audits for us. It saves a lot of time.”

Currently, the AI models used are primarily from Anthropic, and enterprise editions are deployed in a private cloud on AWS, with all the governance and security layered around it because, ultimately, the business is responsible for what the AI does. “You still own it,” he says. “AI may help you, but if it’s wrong, you own it.”

One of the most transformative projects involves rethinking the entire employee experience, building an AI-powered environment based on user personas.

“That’s something we’re working on and hoping to roll it out later this year for the first persona,” he says. It’s a command center experience, a next-gen AI-enabled user interface that replaces the need for employees to navigate all the work systems directly, he adds.

“It’s the future of how we want to enable work,” he says. “It’s intent- and objective-based, and specific to your persona and role, continuously surfacing priorities that are ranked and role-aware with guidance and recommendations, and the work is prepared and orchestrated.” For example, a sales exec with a territory to run might go into Salesforce and look at a to-do list and follow up on things. But that employee still has to decide how to do that work, and how it fits into their part of the overall sales plan.

“In the future, we want that plan to be the context in the new environment, and orchestrate and curate the work for that sales executive, showing real-time telemetry about how they’re progressing on their objectives,” he says. “It’s a different way of thinking to be more productive and build relationships with clients.”

This means that the business has to rethink and redefine the work that goes into the persona-based work roles, he says, and integrate the process flows across multiple systems.

“We’re trying to make the tech invisible,” he adds. “So the experience is elevated, and the human interaction is more seamless, productive, and more valuable.”

Focus on the bottom line

In an Accenture survey of 3,000 business leaders released in July, 23% of companies report widespread and sustained business value from AI, which is down from 32% earlier this year.

As agentic AI goes from theory, to POC, to pilot, to production, businesses are seeing the real costs of the technology, not just in terms of risks, but the financial costs of agents going back and forth with AI models. The inference costs of any one query might be going down, but the total cost for adding agentic AI to the business process is going up. Gartner reported in August that AI inference costs per agentic workflow will increase more than five times through 2028.

For New York Life Group Benefits, this means a focus on the best outcome, rather than having the best AI model. AT&T is also looking at minimizing agentic costs since the company is currently using 45 billion tokens a day. If all of these were routed to the most cutting-edge model, that would be very expensive, and at some point, the costs outweigh the benefits.

“We have an AI gateway that sits between all our requests and endpoints, and directs the request to the right endpoint,” says AT&T’s Markus. “For example, some requests might not require the latest and greatest AI model, but an older, cheaper one would work just as well.”

AT&T also fine-tunes its own models, and trains small language models, he adds. “We’ve got that muscle built now and are optimizing that whole process.” With the right training and data, a small language model can be just as accurate as an LLM for some use cases.

“So when the super agents call subagents, a lot of them are small language models as well,” he says, “And we’re saving 90% of the cost.” Plus, AT&T negotiates price concessions with its vendors, he says. “We’re getting more mature on this.”

But most companies are lagging when it comes to AI cost control. McKinsey found that AI spending increases nearly four-fold as enterprises move from isolated use cases to widespread AI adoption, and a vast majority of organizations report exceeding their AI budgets.

The problem is that AI usage patterns aren’t predictable, governance is immature, teams don’t know which models or tools they should use, and new AI development tools allow employees to build new applications, workflows, and autonomous agents with little effort — and little oversight.

There are many enterprise processes where cutting-edge AI — or any AI at all for that matter — isn’t the solution.

“I don’t need a non-deterministic airline ticketing system or bank app experience,” says Nicholas Mattei, chair of the ACM special interest group on AI and professor at Tulane University. “There are places where agentic workflows are appropriate and mission-critical processes where they’re not. AI isn’t going to speed up your point of sale system.”