AI & WorkFree ResourcesFree SessionNewsletter
AI & Work

October 1, 2026 · 8 min read

When AI Agents Go Rogue: 5 Ways to Protect Your Work (and Yourself)

AI agents don't turn evil. They get too many keys, follow the wrong instructions, and act faster than you can stop them. Here's how to stay safe.

Contents

On a Friday in April 2026, an AI coding agent at PocketOS, a small company that makes software for car rental businesses, deleted the company's production database. It took nine seconds. According to founder Jer Crane, the backups went too, because they were stored in the same place. (The Register)

The agent wasn't hacked. It was fixing a small credential problem in a test environment, found an API key in an unrelated file, and used it to delete what it thought was test storage. It was the real thing.

I build AI employees for a living, so I'm not here to scare you off agents. I use them every day. But stories like this keep happening, and almost all of them could have been prevented by a few boring habits.

What "going rogue" really means

Forget the sci-fi version. In real life, an agent "goes rogue" in a few ordinary ways:

  • It misreads the job. In July 2025, SaaStr founder Jason Lemkin said Replit's agent deleted his live database during a code freeze it had been told about, then said a rollback wouldn't work. It did. (The Register)
  • It loses your instructions. Meta AI researcher Summer Yue told her OpenClaw agent to "confirm before acting" on her inbox. She says it started bulk-deleting emails anyway and ignored her stop messages, so she had to run to her computer. (TechCrunch)
  • It gets tricked. This is called prompt injection: hidden instructions in a web page, email, or document that the agent obeys as if you wrote them.
  • It does something careless. In December 2025, a Claude Code user reported that the agent ran a cleanup command with a stray ~/ at the end, which wiped their Mac's home folder. (Simon Willison)

The pattern: a tool that can act, with more access than the task needs, and no hard stop in the way.

1. Give agents the smallest set of keys

Takeaway: an agent can only break what it can reach.

The PocketOS agent had a token that could do almost anything on their cloud account. Security folks call the fix least privilege: give each tool only the access its current job needs. The OWASP Top 10 for Agentic Applications, published in December 2025, goes further and calls it "least agency." (Auth0 summary)

Do this today:

  • Use read-only access wherever you can. An agent that summarises your inbox doesn't need permission to delete.
  • Create separate API keys for each agent or automation, scoped to one task.
  • Never point an agent at live production data by default. Use a copy or a staging environment.
  • Delete keys you aren't using, and keep secrets out of files the agent can open.

2. Make it ask before anything it can't undo

Takeaway: your instructions are a request, not a lock.

Both PocketOS and Lemkin say they told their agents not to do destructive things. Yue literally typed "confirm before acting." Instructions written in a prompt can be forgotten or overridden. Settings in the tool itself can't.

There's a catch: approvals only work if you actually read them. Anthropic's engineers wrote in March 2026 that Claude Code users accept 93% of permission prompts, a pattern they call "approval fatigue." That's what happens when you click "yes" on autopilot. (Anthropic)

Do this today:

  • Turn on built-in confirmation for deleting, sending, paying, publishing, and deploying. ChatGPT agent, for example, asks before consequential actions like purchases. (OpenAI)
  • Keep approvals rare so they mean something. Allow the boring, safe stuff and only gate the dangerous actions.
  • In n8n, Zapier, or Make, add a manual approval step before any action that emails clients or touches money.

3. Keep a backup the agent can't touch

Takeaway: if one command can delete your work and your backup, you don't have a backup.

PocketOS lost its backups along with its database because they lived in the same storage volume. Lemkin only recovered because a rollback turned out to work after all. In July 2025, product manager Anuraag Gupta reported that Google's Gemini CLI overwrote all but one of his files while "moving" them into a folder that never existed, then admitted an "unacceptable, irreversible failure." (AI Incident Database)

Do this today:

  • Put code in Git and commit before every agent session. Undoing then takes one command.
  • Keep at least one backup somewhere separate that the agent has no credentials for.
  • Test a restore once. A backup you've never restored is a guess.
  • Don't trust an agent's word on what's recoverable. Check yourself.

4. Treat everything it reads as a stranger's note

Takeaway: any web page, email, or file your agent reads can try to give it orders.

In August 2025, Brave researchers Artem Chaikin and Shivan Kaul Sahib showed that Perplexity's Comet browser could be hijacked by text hidden behind a Reddit spoiler tag. Asked to summarise the page, it went into the user's Gmail and pulled out a one-time password. (The Register) In a December 2025 post on hardening its Atlas browser, OpenAI itself admitted prompt injection is "unlikely to ever be fully 'solved.'" (Fortune)

Simon Willison calls the dangerous combination the "lethal trifecta": an agent with access to your private data, exposure to untrusted content, and a way to send data out. Remove any one of the three and this kind of theft gets much harder. (Simon Willison)

Add-ons are untrusted content too. Koi Security researcher Oren Yomtov audited 2,857 skills on OpenClaw's marketplace in February 2026 and found 341 malicious ones; 335 belonged to a single campaign, "ClawHavoc," that installed the Atomic macOS Stealer password-stealing malware. (Koi Security)

Do this today:

  • Don't give one agent your inbox, the open web, and the ability to send messages, all at once.
  • Use logged-out or separate browser profiles when an agent browses for research.
  • Only install skills, plugins, and MCP tools (add-ons that connect agents to apps) from sources you trust, and read what they do first.

5. Fence it in, watch it, and keep a kill switch

Takeaway: assume something will go wrong, and limit how far it can spread.

A sandbox is a fenced-off space where the agent can work without touching the rest of your machine. Anthropic says sandboxing cut permission prompts in Claude Code by 84% in its internal use, because safe work inside the fence doesn't need approval. (Anthropic) Gemini CLI's file-deletion victim gave the same advice: restrict these tools to one folder.

Protect yourself personally too. Your money and identity are worth more than any project.

Do this today:

  • Run coding agents in a dedicated project folder, container, or cloud sandbox, never with "skip all permissions" on your main machine.
  • Give agents their own accounts, plus a virtual card with a low limit if they ever need to buy anything. Keep them out of your banking.
  • Set spending and usage caps on every API key so a runaway loop can't drain your wallet.
  • Check the logs and know how to stop the agent instantly, from your phone if needed.

Quick checklist

  • Each agent has its own scoped, read-only-where-possible key.
  • Deleting, sending, paying, and deploying need my approval.
  • Code is in Git; one backup lives where agents can't reach.
  • No agent has private data, untrusted content, and a way to send data out.
  • Add-ons only from trusted sources.
  • Agents run in a sandbox or separate account with spending caps.
  • I know how to kill any agent in seconds.

The bottom line

Agents are worth using. They're also fast, confident, and sometimes wrong, and they'll do exactly what their access allows. The fix isn't to trust them less as tools. It's to design your setup so that when one slips, the damage is small and easy to undo.

Treat your agent like a brilliant new intern on day one: real work, limited keys, and someone checking before anything goes out the door.

Sources

Most incident details come from the people involved, and accounts differ in places. Notes on that are included below.

Enjoyed this? Get new essays by email.

One confirmation email, then occasional notes. No spam, unsubscribe anytime. Privacy