With AI agents poised to take over significant portions of enterprise workflows, IT leaders will be faced with an increasingly complex challenge: managing them.
“There is a pending concern about how to manage AI agents in the cloud,” says Dave McCarthy, research vice president at IDC, noting that the expanding availability of AI agents from startups and established vendors will give CIOs asset management, security, and versioning challenges.
Analysts say the big three hyperscalers and cloud management vendors are aware of the gap and are working on it. McCarthy, for example, points to the announcement of Google Agentspace in December to meet some of the multifaceted management need. A flurry of innovators, including AgentOps and OneReach.ai, as well as Fiddler, Arize, Pezzo, Helicone, and AimStack, have jumped in to serve the AI agent management needs of enterprise customers today.
“It’s an emerging field,” says Tom Coshow, senior director analyst of AI at Gartner. “If I am a large enterprise, I probably will not build all of my agents in one place and be vendor-locked, but I probably don’t want 30 platforms. What is needed is a single view of all of my AI agents I am building that will give me an alert when performance is poor or there is a security concern.”
While the agentic AI market is in its early days, CIOs — who are optimistic about the promise of AI agents — are aware of the need for managing and monitoring AI agent workflows in real-time.
“Managing agentic AI is indeed a significant challenge, as traditional cloud management tools for AI are insufficient for this task,” says Sastry Durvasula, chief operating, information, and digital Officer at TIAA.
Agentic AI systems require more sophisticated monitoring, security, and governance mechanisms due to their autonomous nature and complex decision-making processes. Current state cloud tools and automation capabilities are insufficient to handle the dynamic agenting AI decision-making.
Durvasula also notes that the real-time workloads of agentic AI might also suffer from delays due to cloud network latency.
“To support business needs, organizations must invest in advanced AI-specific management tools that can handle dynamic workloads, ensure transparency, and maintain accountability across multicloud environments,” he says. “This approach will help businesses maximize the benefits of agentic AI while mitigating risks and ensuring responsible deployment.”
The challenge, however, will be compounded when multiple agents are involved in a workflow that is likely to change and evolve as different data inputs are encountered, given that these AI agents “learn” and adjust as they make decisions.
“If agents are using AI and are adaptable, you’re going to need some way to see if their performance is still at the confidence level you want it to be,” says Gartner’s Coshow.
Agentic AI means constant flux ahead
The dynamic nature of autonomous decision-making means serious guardrails and governance must be in place to prevent unintended consequences, contends PagerDuty CIO Eric Johnson.
“While AI agents can generate significant boosts to efficiency, they can also introduce complexities around AI oversight and accountability. It is important for organizations to establish clear frameworks that help prevent their AI agents from putting their cloud operations at risk, including monitoring agent activities to ensure compliance with data regulations,” he says. “Building trust through human-in-the-loop validation and clear governance structures is essential to establishing strict protocols that guide safer agent-driven decisions.”
Johnson adds that this area is still maturing on cloud management platforms, as well as inside legal, security, compliance teams.
Abhas Ricky, chief strategy officer of Cloudera, recently noted on LinkedIn the cost challenges involved in managing AI agents.
“Developers want to build multi-step agent workflows without worrying about runaway costs. There are organizations who spend $1 million plus per year on LLM calls,” Ricky wrote. “Agent ops is a critical capability — think Python SDKs for agent monitoring, LLM cost tracking, benchmarking, to gain visibility into API calls, real-time cost management, and reliability scores for agents in production.”
As the number and variety of AI agents grow faster than enterprises can consume, so do the requirements for all aspects of management.
Reed McGinley-Sempel, CEO of identity platform Stytch, for instance, notes that the “agent experience” is emerging as a key priority for enterprises, while in the past it was the UX (user experience) and DX (developer experience) that determined how humans and developers interacted with software. The wide range of AI agents — from copilots to coding tools to autonomous assistants — compounds how enterprise CIOs will ensure agentic AI workflows are monitored and managed properly, he says.
“Unlike traditional user authentication, where identity is tied to an individual, AI agents act on behalf of users — raising new questions about trust, permissions, and security boundaries,” McGinley-Sempel says. “If applications do not evolve to accommodate agent workflows, businesses risk either blocking valuable automation or opening themselves up to unauthorized access. The companies that establish clear, standardized authentication flows for AI agents will be the ones that lead in this new era of automation.”
A work in progress
Analysts note that while cloud hyperscalers are working to address the concern it is not known the extent to which Google’s Agentspace, Microsoft Azure AI Services, xAI, or OpenAI’s management platform will offer support, let alone for all agents. This opens the door for a new crop of startups, including AgentOps and OneReach.ai.
Jim Liddle, chief innovation officer for AI and data strategy at hybrid-cloud storage company Nasuni, questions the likelihood of large hyperscalers offering management services for all agents.
“The top challenge with agentic frameworks is that each vendor takes a fundamentally different approach to agent architecture, state management, and communication protocols. As vendors push their own frameworks and agents, enterprises will face adoption challenges, including a significant rise in technical debt and maintenance overhead,” Liddle says. “I don’t see a single framework emerging to unify all agents, making this complexity an ongoing reality.”
Meanwhile, enterprise vendors who are rolling out agentic AI services as part of their flagship offerings make their pitch for a platform-based approach to managing agents.
“Managing agentic AI at scale is a multidimensional challenge. To break it down, it is a governance, operational, ethical, and integration challenge all at once,” says Chris Bedi, chief customer officer and enterprise AI advisor at ServiceNow. “To manage agentic AI, one needs a platform that can unite AI agents, data, and workflows, with a single data model, which brings AI to every corner of the enterprise and addresses all these challenges.”
Either way, Debojyoti Dutta, vice president of engineering at Nutanix, says it will require the entire C-suite to make it all work together and safely.
“We are in the early stages of the multi-agentic transformation of the enterprise. Today, each AI agent is being built and operated in a bespoke way. This will lead to an operational headache for the C-suite,” Dutta says. “The CIO needs tools and infrastructure to operate these agents, the CDO needs to ensure proper governance of data, the CLO needs to ensure compliance and governance of AI in general, the CISO needs to ensure steps to protect the enterprise. The CAIO, along with the other CXOs, needs to work together to ensure that this agentic wave yields actual business returns of this AI investment while protecting the enterprise and keeping the costs in check.”
While agentic AI is still a nascent technology, Gartner’s Coshow says there are CIOs who are already concerned about this complex issue today.
“CIOs are being very cautious and thoughtful about deployment because there are security, government, governance, accuracy, and performance issues that still need to be resolved,” he says.
See also:
Read More from This Article: IT leaders brace for the AI agent management challenge
Source: News