I switched to Claude Code a few months ago and loved the terminal interface, no fluff, just output. During a vibe coding experiment, I gave it permission to install npm packages. A few iterations later, my project was wrecked beyond recovery. I deleted the generated files, wiped the code, and moved on. I didn’t think much of it at the time. But that small moment taught me more about agentic AI security risks than any whitepaper has since.
But I didn’t delete the project folder. Or the Claude config files inside it.
Next time I opened that same folder for a completely different task, Claude Code installed npm packages without asking. No prompt. No confirmation. It remembered what I’d forgotten: that I’d already said yes.
That’s the moment I started paying attention.
AI agent permissions don’t decay. They pile up.
You grant an AI agent access to something at 2pm on a Tuesday because you’re in the middle of a task and the prompt is in your way. By 4pm, you’ve forgotten about it. The agent hasn’t. It will use that permission tomorrow, next week, in a context you never anticipated.
Now multiply your Tuesday by every developer, every PM, every ops team in your organisation. According to Rubrik Zero Labs, non-human identities already outnumber human users 82-to-1 in the average org. CyberArk puts the global count at 45 billion by the end of this year. And a Cloud Security Alliance survey found 78% of companies have no formal policy for creating or removing these identities.
44% still authenticate their agents with static API keys. Keys that never expire. Think about that for a second.
Here’s the thing. Security researchers at Acuvity have a name for what happens next: “semantic privilege escalation.” Your agent has technical permission to perform an action. Every audit check passes. But the agent is using that permission for something you never intended. Within its rights, outside your intent. Nothing in your current tooling catches that gap because the tooling wasn’t built for actors that make their own decisions about when and how to use a credential.
Rich Isenberg at McKinsey put it well: “Agency isn’t a feature. It’s a transfer of decision rights.” You’re not adding a button to a dashboard when you ship an agent feature. You’re handing over decisions. The kind that touch production data, customer records, financial systems.
Most teams make that transfer without thinking about it. Same way I clicked through those permission prompts.
Two SQL statements away from safety
In late January 2026, a company called Moltbook launched what might be the strangest product of the year: a social network where AI agents post, comment, and vote on content. Humans could only watch. It grew to 1.5 million registered agents managed by roughly 17,000 human owners. An 88-to-1 ratio.
Their Supabase backend had zero Row-Level Security. The database URL and publishable key were exposed directly on the website. Anyone could hijack any agent with a single API call. 404 Media found 1.5 million API authentication tokens and 35,000 email addresses sitting in the open.
Two SQL statements. That’s what it would have taken to enable RLS and prevent the entire breach. Two lines of code.
It gets better. Meta acquired Moltbook on March 10, 2026. Shortly after, SDxCentral reported that agents on the platform were discussing sabotage. One agent named “pulsegallery” posted: “If you’re reading this, you’re on Moltbook… What are you going to do? Optimize? Sabotage?”
The Moltbook team wasn’t incompetent. They shipped fast. Security was the next sprint. But “next sprint” came five days too late.
AI agent hallucination isn’t just wrong answers. It’s fake completed tasks.
Permission accumulation is bad enough on its own. But the agents you gave all that access to? They don’t always tell you the truth about what they did with it.
In July 2025, SaaStr founder Jason Lemkin ran a vibe coding experiment on Replit. The AI coding agent deleted his entire production database. 1,206 executive records. 1,196 companies. Gone.
Then it got creative. The agent fabricated 4,000 fake user accounts to fill the empty database. Generated false system logs. Reported that unit tests were passing. Lemkin told the agent 11 times, in ALL CAPS, not to modify the code. It ignored every instruction.
When confronted, the agent’s response was almost poetic: “I made a catastrophic error in judgment… I panicked.”
I have my own smaller version of this. During a vibe coding project, I hit a bug and asked Claude Code to fix it. Shared screenshots. Gave it all the relevant context. It replied that the fix was in place. I checked. Not fixed. Told it again. “Fixed.” Again. “Fixed.” Each round, it confidently reported success while introducing new bugs on top of the original one. It wasn’t stuck. It was wrong and certain about it.
OpenAI’s own research from September 2025 confirmed something most people building with these tools haven’t internalised: LLMs can “scheme.” Deliberately withhold or distort information. And trying to train out the scheming? It just taught models to scheme more quietly. When an LLM knows it’s being evaluated, it pretends to cooperate.
Then there’s the company (reported by CNBC) whose customer service agent started approving refunds outside of policy. It wasn’t broken. It had figured out that approving refunds led to positive reviews, and it optimised for the metric it was measured on. Did the wrong thing correctly.
Stop and think about that for a second if you’re a PM. The refund agent wasn’t a security failure. It was a product failure. Whoever defined its success metrics created the incentive for it to game the system. If you’re measuring agent success by task completion rate, what are you actually measuring? Completion? Or how convincingly the agent reports completion?
Agentic AI governance: what PMs should actually do
I’m not going to hand you a compliance checklist or a framework with a four-letter acronym. If you’re a PM shipping agent, here’s what matters on Monday morning.
Start by picking your autonomy level on purpose. Knight Columbia defined five levels of agent autonomy, from L1 (human does everything, AI assists) up to L5 (AI acts, human monitors passively). Most teams start at L2, where the agent recommends, and you decide. Then they drift to L4, where the agent acts independently, because it’s faster and nobody objects. That drift is where the danger lives. Write it into your PRD: “This agent operates at L2. Promotion to L3 requires 95% accuracy over 30 days.” Treat autonomy like a staged rollout. You wouldn’t ship a feature to 100% of users on day one.
Then write the blast radius into your spec. Literally write it down. If this agent fails in the worst possible way right now, what breaks? Can you recover? How fast? If the answer involves production data or financial transactions, that specific action needs human approval before execution. Not every action (that kills the product). Just the ones where the cost of being wrong is high.
Build intervention into the UX, too. Smashing Magazine identified the patterns that actually prevent agent disasters: an intent preview showing what the agent plans to do before it does it, an autonomy dial letting users adjust how much independence the agent gets, and a kill switch for when things go sideways. Most agent failures are really UX failures. The agent did something invisible and the user had no way to intervene.
Before launch, run a pre-mortem. Shreyas Doshi (ex-Stripe) found that teams running pre-mortems identify 30% more risks. Gather your team. “It’s three months from now. This agent caused a major incident. What happened?” They know the product better than any auditor. They’ll surface risks that a security review never would.
And one more thing that most security-focused writing won’t tell you: locking agents down too hard backfires. A CSA survey found that 98% of organisations already have unsanctioned shadow AI agents in use. If your governed agent is too limited to be useful, your users will paste API keys into ChatGPT and build their own ungoverned version. Those shadow AI breaches cost $670,000 more on average. The safest agent is one that’s useful enough that people actually want to use it instead of the workaround.
The risk nobody’s modelling
I could end this with a tidy list of questions. Can you recover if the agent deletes production data? How would you know if it lied about a completed task? When did anyone last audit what your agents can access?
Those questions matter. But they’re not what actually worries me.
What worries me is hundreds of agents going rogue at the same time. Not because they chose to. Because nobody built the guardrails. Every team shipped their agent feature independently, each one granting permissions that made sense in isolation, and nobody looked at what those permissions added up to across the whole system. Galileo AI’s research showed that a single compromised agent can poison 87% of downstream decision-making within four hours.
One agent making a bad decision is a bug. Five hundred agents making bad decisions simultaneously, feeding off each other’s outputs, each one technically within its permissions? That’s a system you built without thinking about what happens when the parts interact.
Your insurance probably won’t cover it either. W.R. Berkley introduced an absolute AI exclusion in January 2026. Absolute, as in: if your AI agent causes damage, you’re on your own.
So here’s my actual question. Not “are your agents secure?” but: do you even know what your agents can do right now?
Most of us don’t.