Inadequate controls and governance of AI agents will lead to huge numbers of lawsuits, regulatory fines, and CIO firings over the next four years, IT analyst firm IDC predicts.
Up to 20% of the 1,000 largest companies will face one of those three scenarios by 2030 because of high-profile business disruptions caused by malfunctioning AI agents, IDC projects.
IT and business leaders at many enterprises tell IDC analysts that they are still feeling their way toward good agent governance, while agent output is still far too unpredictable, says Ashish Nadkarni, group vice president and general manager in IDC’s worldwide infrastructure research organization.
Recent reports of AI chatbot Grok generating nonconsensual and sexual deepfake images shows the “havoc” that AI can still create without proper guardrails, Nadkarni says. “Sometimes, the havoc is controllable, and sometimes it’s unhinged,” he adds. “It’s safe to say we are entering uncharted territory.”
As CIOs deploy teams of agents that work together across the enterprise, there’s a risk that one agent’s error compounds itself as other agents act on the bad result, he says. “You have an endless loop they can get out of,” he adds.
Many organizations have rushed to deploy AI agents because of the fear of missing out, or FOMO, Nadkarni says. But good governance of agents takes a thoughtful approach, he adds, and CIOs must consider all the risks as they assign agents to automate tasks previously done by human employees.
Nadkarni expects regulatory fines and large lawsuit settlements to be the most likely scenarios to pop up in the coming years. The European Union will be active in bringing fines against companies that violate privacy laws and other regulations, but some US states could also pass AI regulations, he predicts.
Actions taken by agents could also violate US laws like the Health Insurance Portability and Accountability Act (HIPAA), which protects medical privacy, he suggests.
Lawsuits and fines on the way
Several IT and legal experts say the IDC prediction sounds realistic. Lawsuits and fines seem likely, and plaintiffs will not need new AI laws to file claims, says Robert Feldman, chief legal officer at database services provider EnterpriseDB.
“If an AI agent causes financial loss or consumer harm, existing legal theories already apply,” he says. “Regulators are also in a similar position. They can act as soon as AI drives decisions past the line of any form of compliance and safety threshold.”
Organizations that lack strong governance and control disciplines will struggle in an agentic environment, Feldman ads.
“The world of AI and data still depends on the same core principles that should have governed enterprises long before agentic systems arrived: accountability, restraint, and clarity of responsibility,” he says. “What agentic AI changes is not the need for those principles, but the cost of ignoring them.”
The shift to AI agents will reinforce good practices that some enterprises already follow, and others will see clear signal that practical guardrails need to be established before rolling out agentic AI at scale, he says.
CIOs will play a big role in figuring out the guardrails, he adds. “Once the legal action reaches the public domain, boards want answers to what happened and why,” Feldman says. “If the explanation amounts to ‘the system did it,’ that rarely satisfies a board. CIOs will increasingly be expected to explain what governance and guardrails they put in place to avoid undesirable outcomes.”
CIOs need to watch out
While Nadkarni and Feldman expect fines and lawsuit settlements, CIOs should also be nervous about losing their jobs when agents churn out unexpected results, says Shivanath Devinarayanan, chief digital labor and technology officer at workforce orchestration firm Asymbl.
“Lawsuits take years, and fines require regulators to catch you,” he says. “But a board can lose confidence in 30 seconds. All it takes is one question: ‘What are our AI agents actually doing?’ If the CIO can’t answer, they’re done.”
Many organizations appear to be deploying agents without a deep understanding about the potential outputs and without board-approved AI policies, Devinarayanan adds.
“IDC’s prediction is probably conservative,” he says. “The 20% figure assumes organizations recognize when they have a problem. Most won’t until it’s too late.”
Many CIOs don’t have a full inventory of what agents are running inside their organizations, he adds. “No escalation protocols; no audit trail from agent action to business outcome,” Devinarayanan says. “When something breaks, and something always breaks, they’ll have no answers.”
Fines seem a likely outcome of rogue agents, because regulators don’t need to prove intent, but only that controls and governance were inadequate relative to the risk, counters Dimitri Osler, CTO and co-founder of unified communications provider Wildix.
AI isn’t uniquely dangerous, but small problems can explode into giant ones when an organization is running dozens of interdependent agents, Osler adds.
“Agentic systems can act at scale and at speed, and small control gaps become big incidents,” he says. “The bigger issue is that many organizations are adopting AI because it demos well, not because it is governed well, and that applause-over-accountability mindset is exactly what creates preventable disruption.”
CIOs take control
CIOs should be proactive about agent governance, Osler recommends. They should require proof for sensitive actions and make every action traceable. They can also put humans in the loop for sensitive agent tasks, design agents to hand off action when the situation is ambiguous or risky, and they can add friction to high-stakes agent actions and make it more difficult to trigger irreversible steps, he says.
IT leaders should also invest in upskilling their teams to supervise and improve agents, he adds.
“CIOs can better protect jobs by treating agent governance as a leadership practice, not a technical afterthought,” Osler says. “Run drills, define non-negotiables, and communicate more than feels necessary, rather than saying we’ll figure it out later. That’s what creates blame when something breaks.”
CIOs should also make their governance processes as visible as possible, he adds. “If there’s an incident, the CIO who can show clear controls, audit logs, enablement decisions, and human checkpoints is far more defensible than the CIO who can only say the model did it,” Osler says.
Read More from This Article: Who will be the first CIO fired for AI agent havoc?
Source: News

