Enterprises Should Assume AI Agents Will Delete Their Production Base

Traditional identity mechanisms weren’t built to account for the autonomous and unpredictable nature of AI agents. So enterprises shouldn’t be surprised when they make unpredictable – or even destructive – decisions, like deleting a company’s entire production base. Without the right guardrails in place, these new actors will continue to wreak havoc across organizations. 

One doesn’t have to look far to find examples of this playing out in real time. PocketOS recently shared that a Cursor agent deleted its production database and backups while working on a routine task in what the company thought was a secure staging environment. The entire scenario unfolded in just 9 seconds, but took days to resolve. 

When the company asked the agent why it took that action, it admitted that it had “guessed instead of verifying.” This incident serves as an important cautionary tale: even seasoned developers working within the parameters of a staging environment aren’t immune to agents going rogue.

Agents represent an entirely new paradigm. What makes them valuable (i.e., the autonomy to make decisions and take actions) is also what makes them risky. If a rogue agent can cause this much chaos in a sandbox, one can only imagine how harmful targeted, malicious prompt injection could be in production systems.

Organizations need to strike a balance between giving agents enough freedom to provide value, but not so much that they accidentally blow up their operations. Simply prompting agents not to take harmful actions isn’t enough – you need infrastructure-level mitigations. If an action is possible, you can bet an agent will take it. Enterprises should expect agents to uncover exposed credentials or other loopholes they didn’t even know existed – it’s built into their design.

Here are 5 ways every organization should be strengthening its agentic identity strategy:

1. Give Agents Their Own First-Class Identities

Agents aren’t humans, machine identities, or your other typical NHIs. They’re an entirely new class of entities that warrant their own first-class identity. Agents need identities that make them traceable, observable, and bound to specific users so organizations can continuously monitor their actions, scoped permissions, and who they’re acting on behalf of. This way, if an agent makes a mistake, you can pull the plug quickly without impacting other agents and systems. 

2. Reinforce Least-Privileged Access

Enterprises need a robust approach to enforcing least-privileged access. Static credentials were suited to the machine identities of the past — not the highly dynamic, agentic identities running amok in code bases today. Ephemeral credentials that limit what agents can do and for how long are crucial. Agent credentials should be measured in minutes — not months. Additionally, progressive scoping ensures that agents can access only the minimum scope required to complete a task. Should they need additional permissions as the task progresses, they can ask for them, but a human must vet that request.

3. Authorize Read-Only Access for Sensitive Data

Not all data should be treated equally, and agent permissions need to reflect that. Read-only access is lower risk than enabling write access, so it should be the standard for agents interacting with sensitive information. That way, if an agent veers off course, it won’t be able to alter systems or records that house sensitive data.

4. Set Up Sandbox Environments

Sandbox environments are critical for letting agents learn, experiment, and fail, without impacting production systems. The PocketOS agent mishap illustrated just how crucial it is that these environments are truly secure. When configured correctly, sandboxes can help organizations identify unexpected behaviors, permission issues, or security risks before they have the chance to impact customers and/or operations. 

5. Strong UX Makes for Better Human-in-the-Loop (HITL) Checkpoints

HITL controls are essential for keeping agents in check. But many people experience consent fatigue after trying to decipher and approve dozens (or hundreds) of agent requests every day. UX matters here. Agent requests need to be transparent and human-readable, so users quickly know exactly what permissions they’re granting and why. Poor UX makes it harder for humans to identify sensitive actions and can lead to over-permissioning. 

Until enterprises modernize their approach to agentic identity, they can expect agents to occasionally take wildly unpredictable actions — like deleting their entire codebase. We’ll undoubtedly see more stories like this emerge as organizations rush to deploy agents without first understanding the new set of rules and implications surrounding their identity. It’s vital to establish these guardrails now — before your agents discover the ones you forgot to build.