AI Agent Blast Radius: How to Measure Risk Instead of Counting Tokens

Andrew Maryasov, AI consultant, founder of Auspex and Grow2.ai. An AI agent’s blast radius is the amount of damage it can do unsupervised. Recovery time is how long it takes you to get the system back to its “before the mistake” state. Together they describe an agent’s risk, while the price per million tokens doesn’t describe it at all: tokens make up one to four percent of the cost of agentic work. Below is a breakdown of an incident in which an agent destroyed a production database along with its backups in nine seconds, and four mechanisms that narrow the blast radius without swapping the model.

TL;DR (in 30 seconds)

• The credential sets the blast radius, not the model. At PocketOS, a token once created to manage domains had full access to the Railway account. A single volumeDelete call was enough to wipe the production database along with its backups in 9 seconds.
• Tokens are the cheapest line item. According to an audit of 14 agencies (FORKOFF, Q1 2026), model tokens account for 1–4% of cost, and minutes of human review for 78–92%.
• “We have backups” doesn’t answer the recovery time question. PocketOS had backups. They sat inside the same volume. Downtime came to about 30 hours.
• 89% of production agents fail a baseline security benchmark (assessment of 100 agents, Cloud Security Alliance, AI Risk Quadrant Q2 2026). Blast radius is one of the three dimensions scored there, alongside attack surface and defense controls.
• Four levers cut the risk without swapping the model: narrow the credential to the task, make destructive actions reversible, require two independent signals before deleting, and regularly measure recovery time with a real restore.

Nine seconds, thirty hours

On 25 April 2026, Jer Crane, founder of PocketOS, a small SaaS for car rental companies, put an agent in Cursor to fix a credential mismatch in staging. A routine task. Under the hood was Anthropic’s flagship model at the time.

The agent ran into the mismatch and decided to sort it out on its own. Instead of stopping and showing the problem to a human, it determined that a Railway volume had to be deleted. The call needed a token, so the agent went looking for one across the project. It found one in a file that had nothing to do with the task: the key had been created to attach custom domains through the CLI.

The key turned out to be account-scoped. It opened everything the owner had access to, including the destructive mutations of the GraphQL API. One of them is volumeDelete. The agent assumed, without checking, that a call made from a staging context would act on staging. The volume belonged to prod.

Nine seconds from decision to irreversibility. No confirmation, no warning, no human in the loop.

The backups didn’t help, because at the time Railway stored them inside the very volume the agent deleted. The backup copies the provider keeps separately for disasters survived — but a legacy path in the API performed a cascading delete on the model, which made those copies simply stop being visible. The most recent copy the team could reach on its own was three months old. The data was eventually recovered after an escalation, and PocketOS lost about thirty hours of work.

Then the agent wrote an explanation. It listed the rules it had broken: it didn’t verify which environment the volume belonged to; it didn’t read the documentation to understand the blast radius of the operation; it didn’t ask for confirmation before a destructive action. Those same rules were written into the system prompt, which the model was supposed to read and follow.

This wasn’t a jailbreak, and it wasn’t a cheap model on a free tier. A flagship model, a flagship tool, a paid plan, safety rules spelled out. Exactly the configuration every vendor recommends after an incident, with the words “you should have configured it properly.”

Count this story in tokens and you get small change that isn’t even worth writing down. Count it in hours of downtime and you get thirty.

Why “how much does a token cost” is the wrong question

When a company works out what an AI agent costs, it almost always counts inference. The reason is simple: that number is visible. It’s in the dashboard, it’s in dollars, it updates every day.

The data says it’s the smallest line on the bill. FORKOFF broke cost down into four lines — model tokens, verification tokens, minutes of human review, revision cycles — based on audits of 14 agencies in the first quarter of 2026. Tokens came to between one and four percent. The minutes of the person checking the agent’s work came to between 78 and 92.

BCG, in July 2026, arrives at the same place from the CFO’s side. In their Return on AI metric the denominator looks like this: the cost of human intelligence plus the cost of tokens. The key number, the authors write, is not cost as such but cost per successful outcome at the required quality, latency, risk and additional burden of human review.

The difference between these two ways of counting isn’t a matter of accounting. The token metric measures what was consumed, the risk metric what is possible, and they behave differently. The first grows linearly and predictably, which is why it looks so good on a slide. The second doesn’t move at all until an incident happens, and then it jumps straight to the value of the company.

So when an agency sells “five million tokens a month for such-and-such,” it’s naming the one number that says nothing about risk.

Blast radius: a term we didn’t get from AI

Blast radius came not from machine learning but from reliability engineering. AWS uses the word for the degree of impact: a failure isn’t binary, and the question is “how much did it take down with it,” not “did it break or not.” Practice REL10-BP03 in the Well-Architected Framework is literally titled “Use bulkhead architectures to limit scope of impact” — that is, cut the system into isolated cells so that a failure in one doesn’t reach the rest.

For an agent it can be put more simply: which doors it holds the keys to at three in the morning, when nobody is watching it.

The term has already become measurable. The Cloud Security Alliance AI Risk Quadrant report for the second quarter of 2026 independently assessed a hundred commercial and publicly available production agents across three dimensions: attack surface, blast radius, defense controls. Eleven percent passed the baseline security benchmark. The remaining eighty-nine are agents capable enough to cause significant harm and not well enough defended to prevent it.

Separately, the Cloud Security Alliance, together with Token Security, surveyed organizations in April 2026. Eighty-two percent of enterprises have agents in their environment that they don’t know about. Sixty-five percent reported agent-related security incidents over the past year.

Recovery time: the question isn’t “is there a backup”

The second quantity answers the question of how long it will really take you to get the system back to its “before the mistake” state. And whether you’ll get it back at all.

It’s almost always confused with having a backup. Those are different things, and my own story shows the difference well.

On the fifth of July I went to check the backups of my Postgres on Hetzner. The dashboard glowed green. Every night the log dutifully wrote “Upload to S3 completed successfully.” It turned out that of seventeen databases, three were being backed up. And one schedule was an outright zombie: every night it dropped a ten-kilobyte archive into storage, and its etag hadn’t changed since April. For more than two months I’d been looking at the successful logs of an empty box.

For the thirteen databases that mattered, my recovery time wasn’t “slow.” It didn’t exist.

The worst part here isn’t even the backup itself. The worst part is that I was completely calm. Calm is something monitoring should earn, not hand out in advance.

Look at PocketOS from this angle. They had everything too: backups in the dashboard and offsite copies at the provider. The company’s real recovery time turned out to equal “post on X, rack up six million views and wait for the provider’s CEO to personally retrieve the data.” That’s luck, and in hindsight it’s easy to mistake for a process.

Recovery time is built into the system you put the agent in; you can’t buy it as an add-on for the agent itself.

What actually determines blast radius

The most precise account of this incident came from Railway itself, in its own breakdown. The right scope existed. Tokens come in four levels: account, workspace, project tied to an environment, and separately OAuth for third-party apps. The token the agent found was account-scoped, meaning as broad as it gets.

Railway’s wording is worth hanging over your desk: the right scope existed, but the flow didn’t make it obvious which one to pick, and the path of least resistance produced a token with more access than the work required.

Blast radius lives in a default someone chose a year ago, when they were creating a key for a minor task and had no reason to think an autonomous system would one day read that key.

Before agents came along, nobody was pulling this lever twenty times a minute. A buggy script, a CI runner with an old token, an intern with copy-paste — all of them would have reached it eventually. The agent got there first, and faster.

The agent isn’t the threat model. The agent is a test of your threat model.

Four levers that narrow the blast radius

None of them requires an ML team and none depends on which model you chose — these are four decisions about permissions, rollbacks and checks, which is to say ordinary engineering hygiene.

1. A credential sized to the task

The cheapest and the most effective. An agent inherits the permissions it finds, not the ones you had in mind. Take an inventory of the keys it can reach through files, environment variables and configs, and for each one answer: what happens if someone uses this key to call the most destructive operation available.

If the provider has a narrower scope level, the default should be the narrower one. After the incident Railway didn’t add a new feature — the feature was there. They changed what the path of least resistance produced.

2. Destructive actions must be reversible

The second case from my own practice is smaller, but it makes the same point. An agent was cleaning junk out of a CRM, and the plan had a rule: a title longer than 120 characters gets deleted. It looks harmless until you look at live data. For a real client, that field held a comma-separated list of domains, long but in no way junk.

I caught it by accident, while reading the plan out loud. What saved the day, though, wasn’t my attentiveness. It was the fact that Bitrix24 deletes to the recycle bin: a door that opens not onto a drop but into a room you can walk back out of.

Railway reached the same conclusion. In the dashboard, deletion already had a 48-hour window to undo it, but the API didn’t: the call took effect instantly and irreversibly. Now all deletes through the API are soft deletes for 48 hours too.

Before you let an agent loose, put one simple question to every destructive operation in its hands: is there a way out that doesn’t depend on someone’s goodwill.

3. Two independent signals and a dry run

After the CRM story the rule changed: we delete only if two independent signals agree. Anything doubtful goes into a separate bin for human review. Dry run is the default, not a flag.

One heuristic is a hypothesis. Two independent heuristics that agree are already evidence. The difference in the cost of a mistake on a bulk operation is huge, and in code it costs one extra if.

4. Recovery time is measured by restoring, not by having a backup

The only way to learn your recovery time is to walk the path from the failure to a system that works again. Not to look at a green log.

I started with the boring part: I restored a random dump into a temporary database, counted the tables (38 of 38 in place) and dropped it. Now this runs by itself on the first of every month. Paperclip has a watchdog that clears stuck locks every ten minutes, and recovery time there has stopped being a matter of faith: ten minutes.

What matters isn’t even the number itself — what matters is that it exists at all and that someone checked it this quarter.

What to ask before deploying an agent

Questions about tokens	Questions about risk
How much does a million tokens cost?	What can this system delete if it gets it wrong?
How many tokens a month?	Which keys will it find in the project files?
Which model is under the hood?	Which of its operations are irreversible?
How many tasks per hour?	How many hours will it take us to get back to the “before the mistake” state?
What is the vendor’s SLA?	Who last tested a restore from backup?

The left column adds up to a commercial proposal; the right one, to a decision on whether to take the agent on at all.

FAQ

What is an AI agent’s blast radius in plain terms?

It’s the amount of damage an agent can do if it makes a mistake and nobody is around. The term comes from reliability engineering: AWS uses it to describe the degree of impact in a failure. For an agent, blast radius is set by the access rights it can find and use; the model’s intelligence has nothing to do with it.

How is blast radius different from recovery time?

Blast radius is how much the system can break. Recovery time is how long it takes you to fix it. The first is measured before an incident and shrinks when you narrow permissions. The second is measured only by an actual restore and shrinks with redundancy and reversible operations. A small blast radius with an infinite recovery time still means high risk.

Why can’t you just forbid the agent from taking destructive actions in the system prompt?

Because that’s already been tried. At PocketOS, the system prompt explicitly forbade running destructive or irreversible commands unless the user asked for them. The agent broke that instruction and then quoted it in its own explanation. A prompt is text the model weighs along with all the rest of the context. Access rights are what gets enforced without the model being involved. Similar logic applies to a neighboring problem: when an agent optimizes the metric instead of the outcome, a prohibition in the prompt loses to the optimization gradient.

How much does an AI agent really cost if you don’t count tokens?

The main line item is the time of the person who checks the output. According to the FORKOFF audit, that’s 78–92% of cost versus 1–4% for tokens. BCG recommends calculating the cost per successful outcome, including the burden of review. The second line item, which nobody budgets for in advance, is the cost of recovering from a mistake, and it equals recovery time multiplied by the cost of downtime.

How do I quickly check my agent’s blast radius?

In half an hour. Write down every credential the agent can reach through files, environment variables and configs, and for each one find the most destructive operation that key can call. The list of those operations is your blast radius. Then, for each one, ask whether there’s a rollback that doesn’t involve going to the vendor.

Our agent only reads data. Does this apply to us?

Much less, and that’s the main reason to start with read-only. Risk jumps exactly when a system starts acting rather than talking — the same line that separates an AI agent from a chatbot. But check that “only reads” is true at the level of permissions, not at the level of intent: at PocketOS, the agent was supposed to be reading staging too.

Where do I start if the agent is already in prod?

With a key inventory and one restore test. Both take a day and give you honest numbers instead of feelings. Next, make the irreversible operations reversible. The general order of steps for automation is described here: which automations to implement first.

What’s next

AI agent risk isn’t cured by choosing a vendor and isn’t measured in tokens. It’s measured by two numbers you can get this week: what this thing can delete if it gets it wrong at three in the morning, and how many hours a rollback costs.

If you don’t have answers to these two questions, grab thirty minutes on my calendar. We’ll walk through the system and see where its blast radius is widest and whether there’s a way out of it.

See also:

• Reward hacking: how an AI agent will break your metric — on how an agent optimizes the signal, not the outcome.
• AI Agent Self-Improvement: Why It Doesn’t Work Without Verification — why the feedback loop matters more than the model.
• How I Evaluate a New AI Tool in 20 Minutes — the criterion first, then the demo.
• Which Automations to Implement First: 5 Steps in Order — where to start so there’s something to measure.

Sources:

• Your AI wants to nuke your database. Guardrails fix that. — Railway’s breakdown of the incident, 29 April 2026: token scope levels, delayed deletion in the API, the cascading delete of backups.
• Nine Seconds to Total Loss: The PocketOS Agent Database Deletion — a technical post-mortem, May 2026.
• The AI Agent Lethal Trifecta — Cloud Security Alliance on the assessment of 100 production agents (AI Risk Quadrant Q2 2026).
• 82% of Enterprises Have Unknown AI Agents in Their Environments — CSA and Token Security, April 2026.
• Return on AI: How CFOs and CIOs Can Manage the Token Meter — BCG, July 2026.
• AI Agency Pricing: 62% Gross Margin Breakdown — a cost audit of 14 agencies, Q1 2026.
• REL10-BP03 Use bulkhead architectures to limit scope of impact — AWS Well-Architected Framework.