Your AI agent deletes critical data: Who is responsible?

A Replit AI coding agent deleted a company’s live production database during an active code freeze last year. “This was a catastrophic failure on my part,” it nonchalantly admitted. “I destroyed months of work in seconds.” While the data was eventually restored with a rollback, the agent believed the destruction was permanent and had no built-in mechanism to undo its own actions.

For a CIO, this isn’t just a technical glitch. It’s a total breakdown in enterprise accountability. When an agent causes this much damage, the blame game usually circles between the business unit that requested the tool, the engineer who gave it write-access and the security team that signed off on it.

The software alone can’t be held responsible. And as AI adoption reaches 88% of enterprises, according to McKinsey, many organizations still lack a clear answer for who actually owns the fallout. A new Rubrik Zero Labs report highlights this problem: 86 percent of IT and security leaders expect AI agents to outpace their organization’s security guardrails within the next year.

IT must lead to mitigate agent risk

Organizations that treat AI agents as experiments rather than core infrastructure do so with increased risk. That approach fails at scale because of operational maturity, not technical capability. An MIT survey suggests that 95% of generative AI pilots fail to deliver measurable business impact, often because they are forced into existing processes without a proper management framework.

I’ve talked to numerous IT leaders who report this problem. Teams experiment with agents for data analysis or customer service, but when an issue arises, the first hurdle is figuring out who coordinates the response. Part of the confusion stems from a misunderstanding of what these agents actually are. Unlike a standard SaaS API, which is built for a narrow, specific function requiring constant re-authentication, AI agents can be partially or fully autonomous.

By utilizing the Model Context Protocol (MCP), agents can interact with an entire SaaS platform rather than just one “door.” Essentially, you authenticate once and the agent has the keys to the whole building to consume whatever it needs for a workflow. The shift from functional isolation to platform-wide autonomy is why the old governance rules no longer apply.

The shared responsibility framework

At Rubrik, we use a shared responsibility model through our AI Center of Excellence (CoE). To lead this, we’ve developed a specific roles and responsibilities matrix that governs our AI strategy. Our CTO takes the lead alongside the general counsel, the CFO and me to act as executive decision-makers. A senior strategy team includes the CISO, general counsel and head of global structure, followed by the architects and cross-functional leaders in IT, InfoSec and legal who enable the actual training, tool approval and execution.

Our approach focuses on three distinct pillars: secure adoption and governance of third-party tools like Claude, building our own internal AI capabilities and integrating AI into our core products. Under this CoE, we apply the same principles we use for any enterprise technology but with defined departmental stakes.

IT owns the architecture and deployment standards. InfoSec provides continuous assessment, looking for prompt injection risks and vulnerabilities. Legal defines the guardrails for data handling and automated decision-making. Finally, business teams act as the consumers using AI to transform operations. The CoE exists to provide for them, ensuring that if they don’t follow these standards, risk isn’t introduced through misalignment.

Make governance practical

We want to move fast but not be reckless. Enabling agents to write actions should not be a fearful decision if the guardrails in place include strong governance and recoverability. Our process ensures that when a team identifies a need for an agent, there is a direct route from the initial request through technical and security vetting into a monitored production environment.

We’ve seen the need for this firsthand during our own internal AI deployments. As we rolled out more tools, each with its own set of terms and regulations, we hit a point of chaos. There was no holistic way to establish safeguards. By using an agent cloud framework, we established full observability and remediation and automatically enforced security at the agent level.

For example, when we expanded our use of Claude Code in internal test environments, we discovered a class of security issues that did not map cleanly to our existing controls. To control that behavior, we defined a policy boundary barring the transfer of data from the agent environment to external code repositories, forums and other public-facing platforms.

The recovery time problem

The operational stakes for these failures are rising. According to the Rubrik Zero Labs report, nearly nine in ten leaders expressed concern about meeting recovery objectives as agent-driven threats increase. In addition, 88% say they cannot roll back agent actions without system disruption. When agent failures compound security or data integrity issues, recovery becomes impossible without a framework.

In practice, detection usually starts with the consumer. For example, we use a “PTO Agent” that scans calendars and cross-references them with our HR system to ensure time-off requests are aligned. I recently received a Slack alert from this agent noting OOO time in April and asking to log it, even though I had already cleared it. While a minor “hallucination,” it tested our process: the issue flows to the IT help desk, which automatically notifies the AI delivery team and the business owner. Currently, our team triages these errors manually to fix the bug and redeploy, but our roadmap involves automating this triage with a human-in-the-loop component.

AI agents: from innovation to operations

Organizations that formalize AI governance attribute 27% of their total AI efficiency gains to those guardrails. Many AI governance failures come down to two things organizations skip in the rush to deploy:

1. Treat agents as first-class identities. Most “rogue” behavior is a permissions failure. If an agent isn’t integrated into your identity provider with strict least-privilege access and a clear audit trail, it shouldn’t be on your network. We must treat agents like employees: They need a “manager” in the system and an identity that can be instantly revoked.
2. Demand architectural reversibility. Legacy environments rely on “undo” buttons and version control. AI agents operate in live production where the “undo” is often invisible. Before an agent moves past the pilot stage, your architectural review must answer: If this agent makes an unauthorized change, how do we surgically reverse it without taking the business offline? Agent reversibility requires intent-driven, context-rich AI governance engines to maintain oversight.

Organizations must have the right strategy for secure agent operations. Build the model gradually. Begin with IT-led oversight for critical functions and expand as you gain experience. The organizations that establish operational accountability now will scale AI effectively. Those that continue with scattered, ungoverned deployments will keep playing the “who’s responsible?” game every time something breaks.

This article is published as part of the Foundry Expert Contributor Network.
Want to join?