AI interoperability challenges persist, even after new protocols

Within a year, many enterprises may be running dozens of AI agents, some built in house and others purchased from five or six different vendors.

For some CIOs, this may sound like an interoperability headache. A handful of AI standardization protocols have emerged in recent months, but it’s still unclear which protocols will win in the market, and CIOs still face a number of sticky issues.

Over the longer term, advocates of multi-agent IT environments see the potential to drive huge efficiency gains and significant cost savings, as agents take over a host of mundane and repetitive tasks now done by employees. But in the meantime, many IT leaders see some serious interoperability challenges, including data security, cost containment, and a lack of trust in the outcomes when multiple agents work together.

AI interoperability is becoming a major issue for companies looking to tap into the potential of the technology, says Steve Taplin, CEO of software development firm Sonatafy Technology.

“As someone leading AI-driven transformation projects for enterprise clients, I can tell you AI interoperability is quietly becoming one of the most pressing obstacles to scaling,” he says. “We’re seeing firsthand how organizations are struggling not just to adopt AI but make their growing ecosystem of AI tools actually work together.”

When interoperability isn’t addressed at the architecture level, organizations create brittle and unscalable AI implementations, Taplin adds. “Companies end up with siloed models, duct-taped integrations, and rising support costs,” he says. “Worse, their teams lose confidence in the AI they’ve worked hard to adopt.”

Vendors serving organizations rolling out AI agents and related technologies are seeing a new wave of integration problems, adds Ian Stendera, chief product officer at enterprise architecture management vendor Ardoq.

“Right now, we’re seeing the first taste of these potential challenges arise as agents actually start to get a footprint,” he says. “Until now, it’s been a series of failed experiments, but it’s starting to get some traction.”

Competing protocols

The new AI protocols are coming out as many organizations begin to deploy agents at scale, Stendera says, but it’s still unclear which protocols will emerge as winners. Some organizations will pick the wrong horse.

“It’s coming together at the right time, but we’re seeing the need to find some standards, so we don’t accidentally end up in a space where we’re no better off than we were before,” he adds.

Stendera also sees data security as a big issue to be resolved, with many CIOs concerned about which agents are accessing sensitive information. In a related issue, distrust in agentic outcomes will be a particularly difficult problem to solve when several agents work together to generate a report or requisition new supplies, he says.

Agentic AI may soon have a “turtles all the way down” problem, in which a series of agents build on each other’s work to produce an outcome, he adds. In this case, a bad decision or result generated by one agent would contaminate the entire result.

“You might have a hallucination somewhere in the chain that can act like a malicious user as it corrupts that workflow,” Stendera says. “Because of the way this architecture is being designed, with multi-agent orchestration, it’s really hard to maintain transparency and auditability.”

Yet another major problem with interoperability involves the cost of agents, he suggests. At many organizations, separate business units may roll out their own agents, and in some cases, employees or units may build their own, as low-code agent development becomes a thing.

Stendera says that CIOs should ask themselves several questions like how many agents do we have doing the same thing? Are we paying too much? Are we paying too much to one specific vendor?

“Those kinds of things will come in later, but haven’t really bubbled up yet,” he says. “It’s still almost like this endless budget of experimentation. But as we start to see it take hold, we’re going to see these kinds of duplications and redundancies being a real challenge.”

Formatting errors

Beyond security, trust, and cost issues, Sonatafy’s Taplin sees several other AI interoperability problems, including data format chaos and workflow disruption related to what he calls integration fragility.

“Every new AI platform, agent, or service seems to come with its own ‘language’ — data formats, APIs, runtime dependencies, and operational assumptions,” he says. “It’s a modern-day Tower of Babel. Things start to break apart when CIOs try to stitch these tools together, especially across departments or business units.”

It’s a problem because all AI tools don’t handle data the same way, he says. For example, a logistics client of Sonatafy has deployed three AI tools for forecasting, routing, and prices, with each using slightly different timestamp conventions and regional encoding. These subtle differences led to misrouted shipments, inventory misalignments, and customer service problems, Taplin says, even though the systems were supposed to be interoperable.

“We’re finding that many AI tools — even when built to interact with standard datasets — encode or process data differently under the hood,” he adds. “This leads to subtle mismatches that cause failure in chaining models or agents together.”

In addition, workflow disruptions can happen through software updates when organizations have deployed several AI tools that work together, Taplin says.

“CIOs want automation pipelines that can adapt over time,” he says. “However, when you patch together multiple AI services from different vendors or open-source models, even minor updates can collapse the entire chain.”

Taplin also witnessed problems when a marketing team deployed an LLM-powered campaign engine tied to a sentiment analyzer from another vendor. When the analyzer was updated with a new model version, it started classifying inputs in a new way, breaking personalization logic downstream, he says, and no one caught the problem until the marketing campaign failed.

CIOs should acknowledge that AI integration isn’t just about APIs, he adds. “It’s about shared expectations, version transparency, and behavioral predictability, all of which are still sorely lacking across many tools,” he says.

Dealing with the challenges

To fix potential interoperability problems, CIOs need to take a thoughtful approach to deploying agents and other AI tools, Taplin says. Protocols allowing AI tools to talk to each other aren’t enough.

“Industry standards are a good first step, but they don’t solve the underlying tension between rapid innovation and long-term maintainability,” he says. “You still need internal engineering rigor and a willingness to slow down just enough to design AI systems that play well with others.”

Organizations should consider a centralized platform approach to deploying AI agents instead of allowing separate business units or individual employees to launch their own, adds Sunil Senan, SVP and global head of data, analytics, and AI at IT consulting firm Infosys.

This platform approach, done correctly, can anticipate many of the trust, risk, governance, and other potential problems related to AI interoperability, he says.

“By doing the platform-based deployment, you bake in your responsible AI principles,” he adds. “We support the idea of having a well thought out and responsible AI process that supports agent AI integrations and, in terms of operability, cuts across applications, but is governed through the platform.”

Senan also advises CIOs to consider agents that can handle several tasks across multiple applications, instead of stringing several agents from different vendors together to assist an employee. For example, a business analyst in the oil and gas industry may work with a single agent to summarize industry reports from PDFs, process data from the company’s SAP system, and interact with Microsoft’s Office suite, instead of using three agents.

“You can see how I’ve transcended boundaries across the application landscape to make the agent meaningful for the role I play, as well as make it support me, or even displace work I do so I can focus on more high value tasks,” he says. “An agent may not be limited to a particular application.”