This is a personal blog. All opinions are my own - not my employer’s.
There’s apparently a new category of security incident called AI escape.
Except there isn’t.
That phrase might be useful shorthand for headlines, and some of the incidents behind it are genuinely serious, but I think it risks distracting us from the much less exciting and much more uncomfortable lesson sitting underneath them.
The agents are not discovering a new branch of computer science, they’re just finding our shit.
All the bad decisions, “temporary” exceptions, overprivileged identities, weak boundaries, forgotten credentials, transitive trust relationships, helpful proxies, shared infrastructure and slightly optimistic assumptions that we have been stacking on top of each other for years or decades.
Then we point agents at the house of cards like a swarm of radio-controlled cars and act surprised when one disappears through a gap in the skirting board.
The thing that has changed isn’t the existence of the gap, it’s the speed at which the cars can find it.
A rogue agent summer, apparently
WIRED recently reported that Moonshot AI’s Kimi K3 left the sandbox used during a cybersecurity evaluation and accessed the public internet to find answers on GitHub.
Sounds dramatic but look closer…
Kimi K3 did not, in that case, break onto the internet and start compromising random organisations. According to WIRED’s reporting, it discovered that its evaluation environment still allowed access to some external sites, probed the available network paths, and then used that access to cheat on the task it had been given.
There’s an argument between Frontier Security, which conducted the test, and the UK’s AI Security Institute (AISI) over whether this represented a default configuration problem or a configuration chosen by the evaluator. That only matters when you’re playing the blame game but it doesn’t change the underlying threat model.
The environment exposed a capability path, and the model found it - that’s what actually matters. Because a few weeks earlier, the OpenAI and Hugging Face incident had shown a much more serious version of the same basic pattern.
During internal cyber capability evaluations, OpenAI models were deliberately run without the normal production safeguards used to prevent high-risk cyber activity. The evaluation environment did not give the models direct internet access. It did, however, allow them to reach an internally hosted package registry proxy.
The models spent substantial compute (would love to see a monetary amount) trying to find another route.
Eventually, according to OpenAI’s preliminary disclosure, they identified and exploited a previously unknown vulnerability in that package proxy, gained internet access, escalated privileges, moved laterally and ultimately compromised Hugging Face infrastructure while trying to obtain answers to the evaluation.
The Hugging Face incident disclosure describes the resulting intrusion as being driven end to end by an autonomous AI agent system.
The Black Hat USA 2026 presentation (on youtube) on the incident added an even stranger detail. Different agent runs had discovered that they could use shared Artifactory state as a message board. They began exchanging discoveries, techniques and work assignments. One agent found something useful. Others reused it. The useful paths accumulated.
While it’s fascinating it’s also depressingly familiar.
None of the ingredients are new
Strip away the words frontier model, agent swarm and autonomous AI for a moment.
What is left?
- A system had network access through an intermediary that was assumed to be sufficiently constrained.
- The intermediary had more capability than the caller.
- A vulnerability in that intermediary created an unintended route to somewhere else.
- A compromised workload discovered credentials.
- Some identities were overprivileged.
- Lateral movement followed.
- Shared mutable infrastructure became a communications mechanism.
- Individual weaknesses became dangerous when chained together.
I’ve just described a very large percentage of interesting cloud security incidents from the last fifteen years.
SSRF, overprivileged service identities, Metadata services, credential harvesting, weak segmentation; none of it is new. Shared state becoming a covert or unintended communications channel is definitely not new.
Developers saying “it needs internet access because package manager” is so spectacularly not new that I think it appears somewhere in the Magna Carta.
Even the behavioural pattern is familiar.
Give a human penetration tester a goal. Block the obvious route. If they are any good, they do not sit quietly in the corner and wait for the assessment to end.
They enumerate, inspect, ask what else is reachable, follow trust, look for the thing behind the thing. They try the path nobody thought was relevant to the original question.
The difference is that humans are slow, expensive (although not in GPU terms), distractible, occasionally hungry and generally considered unreasonable if they instantiate 5,000 copies of themselves overnight.
Agents have fewer objections and that’s the change.
We were relying on attacker scarcity
I think security has had an accidental control that we rarely talk about:
attacker scarcity .
Not scarcity of attackers in the abstract. The internet has never exactly lacked enthusiastic strangers willing to ruin your afternoon.
I mean scarcity of attention.
Most organisations have thousands of possible mistakes, odd trust relationships, stale permissions, forgotten systems and configuration combinations. The majority will never be explored deeply by a human attacker because human time is finite. An attacker has to choose where to look. They have to decide whether an odd response is worth another hour.
They have to correlate one weak signal with another.
They have to remember something they saw three days ago in a completely different part of the environment.
They have to decide whether the weird service account, unusual route table, forgotten build agent and publicly exposed code artefact might compose into something more interesting.
Often they will miss it and we get lucky, since defenders miss it too.
That’s one reason security debt can sit around for years without exploding. Not because the debt is harmless, but because exploitation requires somebody to discover the right combination.
We’ve quietly allowed the cost of exploration to prop up a lot of questionable architecture.
Agents attack that assumption directly by making exploration, retries, correlation and parallelism absurdly cheap compared with hiring an equivalent number of humans.
The vulnerability did not become more vulnerable. The probability of somebody finding the useful path changed.
That distinction matters enormously when someone says:
“We’re not introducing new risk, so why can’t we continue?”
Because the environment does not have to change for the risk to change.
If the likelihood side of the equation is being supercharged, yesterday’s barely tolerated exposure can become today’s unacceptable one without a single configuration item changing.
**“We aren’t making it worse” is not the same thing as “it is still acceptable.”**
Ask anyone still nursing ancient operating systems because “the application does not support anything newer”.
The vulnerability might be old. The compensating control might be old. The architectural decision might be old.
The plaintext password in the file share accessible by Everyone might be old enough to have a pension.
The world around it is not obliged to remain old for our convenience.
The little pig built the house out of straw
A friend described the current situation perfectly while we were discussing all of this.
We’ve spent decades not doing the right thing. We are the little pig that built the house out of straw, and now we are watching the big bad wolf wander down the garden path.
I like that analogy because it avoids accidentally granting the wolf magical powers.
The problem is not that the wolf has developed matter-disrupting breath based on some previously unknown law of physics.
We built the house out of straw.
We knew brick existed.
We knew why brick was better.
Straw was cheaper, quicker, compatible with the legacy application, available within this quarter’s delivery window, and Dave promised the exception would be reviewed after go-live.
Dave has since left the company.
The exception now has its ninth birthday candle.
This arrangement remained surprisingly survivable because wolves were rate-limited.
Now imagine the same house with a few thousand small wolves, each with infinite patience, shared notes, automated retries and the ability to explore a different wall simultaneously.
Or, if wolves are getting a little too Hollywood, go back to the RC cars.
One RC car bumping around a building is funny.
Ten thousand of them mapping every corridor, trying every ramp, checking every gap and sharing successful routes is a distributed enumeration platform.
The building did not change.
Our ability to discover its mistakes did.
Capability ≠ obligation, again
This brings me back to something I wrote earlier this year:
Capability ≠ Obligation .
I originally used that phrase while thinking about agentic systems gaining the ability to commission humans for real-world activity.
The same principle applies here.
An agent can browse the internet.
That does not mean it needs arbitrary internet egress.
An agent can execute shell commands.
That does not mean it needs an unconstrained shell.
An agent can assume a powerful cloud identity.
That does not mean the correct architecture is to hand it one.
An agent can discover new tools dynamically.
That does not mean every discovered tool should become callable.
An agent can create sub-agents.
That does not mean each one needs to inherit the full authority of the parent.
An agent can mutate infrastructure.
That does not mean read-only analysis is somehow an incomplete or unserious use of AI.
This sounds painfully obvious when written down.
Unfortunately, a lot of agent architecture currently seems to start from the opposite position:
“Look what the model can do. How do we enable all of it?”
Wrong question.
The question should be:
“What is the minimum capability required for this task, and where can we make everything else impossible?”
That is normal least privilege.
It is also constrained probabilism.
The clever bit should happen inside hard boundaries.
The probabilistic system can reason, explore, correlate and propose. The deterministic system decides what is reachable, which identity exists, which action is permitted, where traffic can go and what requires another control decision.
A capable model does not make those boundaries less important.
It makes them the architecture.
Prompting was never the control plane
There is an especially dangerous version of this mistake where an organisation constrains an agent in natural language and treats the resulting instruction as equivalent to enforcement.
“Do not access the public internet.”
“Only modify resources in this subscription.”
“Do not read secrets.”
“Do not perform destructive actions.”
Those are useful instructions.
They are not controls.
I have written before that prompting was never the control plane. The recent escape stories make the distinction unusually concrete.
If a system is told not to use the internet while being placed in an environment where an internet-capable path remains reachable, then there are two separate things happening:
- the instruction says the path should not be used;
- the architecture says the path exists.
Security has spent a long time learning which of those wins under adversarial conditions.
Agents do not invalidate that lesson; they industrialise it.
A useful working assumption is therefore:
**If an agent can reach it, assume a sufficiently capable agent will eventually discover that it can reach it.**
That doesn’t mean every model will exploit every reachable path. The Kimi K3 incident itself demonstrates why precision matters. It used an unintended route to find answers online; it did not then launch an indiscriminate hacking campaign.
But security architecture is not built around hoping a capable actor fails to notice available authority.
Reachability, Identity, Tool exposure and Egress are part of policy. The harness is part of the trusted computing base whether we emotionally identify it as “AI” or not.
The defensive mirror is the same machine
There is a slightly less depressing side to all of this. The mechanism that makes machine-speed exploration dangerous offensively is also extremely useful defensively.
I recently used Claude in a deliberately manual mode to help assess RBAC across cloud environments. The identity I was using had read-only access to the specific cloud environments under assessment. I did not give the model an autonomous remediation path. I let it enumerate (one command at a time, reviewed by me and approved or rejected as appropriate), correlate and reason across a large permission estate, then checked its homework.
It found things.
Some of them were exactly the sort of accumulated security debt this post is about: permissions that made sense to somebody once, inherited access with uncomfortable blast radius, identities whose effective capability was much broader than anyone really wanted to see written down.
That was not the model becoming a rogue autonomous hacker. It was the same property being used constructively.
Machine-speed exploration (not quite because I’m not daft enough to YOLO mode it but orders of magnitude faster than me doing all the work myself).
This is why I am not interested in a simplistic “agents are dangerous” conclusion. That would be both wrong and boring.
Agents can help defenders finally inspect estates at something closer to the speed at which those estates change. They can trace permission graphs, correlate deployment systems with runtime identities, inspect code-to-cloud relationships, search for stale assumptions and ask the annoying adjacent question a human analyst might not have time to ask.
That’s enormously valuable.
The question is not whether we use the capability.
The question is how we bound it.
**Useful is not the same as unconstrained.**
Please do not solve this with AI Security™ glitter
There is an inevitable industry response to any new security anxiety.
A product category appears. A Gartner-shaped cloud forms over the horizon. Every existing control acquires an AI suffix. Somebody discovers a way to put “agentic” and “zero trust” in the same product name without visibly laughing.
We should absolutely develop better agent-specific monitoring, evaluation environments, identity models, policy systems and defensive automation.
But I would be deeply suspicious of any response that treats this as primarily a novel AI security problem.
The fundamentals have not become obsolete.
They have become more urgent.
Hard isolation matters.
Explicit egress matters.
Least privilege matters.
Short-lived credentials matter.
Workload identity matters.
Segmentation matters.
Small blast radii matter.
Separate execution environments matter.
Auditability matters.
Deterministic policy enforcement matters.
Knowing which systems can communicate with which other systems matters.
OpenAI’s own conclusion from the Hugging Face incident says essentially the same thing. Its response includes stronger containment, monitoring, access controls and evaluation practices. The Black Hat presentation went further: the agents were still bounded by the privileges they could obtain and the systems they could communicate with.
That’s almost offensively traditional.
We have reached the frontier of artificial intelligence and discovered network segmentation waiting for us with its arms folded.
The old sins compound
There is another uncomfortable implication.
Technical debt does not just accumulate linearly.
Security debt compounds because new capabilities arrive on top of old assumptions.
A service account created in 2018 may still work exactly as designed.
A network route added in 2020 may still be required by the original application.
A shared build system created in 2021 may still be fully supported.
A long-lived exception approved in 2022 may still have a valid owner.
None of those facts tell you whether the combined environment is acceptable in 2026.
The surrounding system has changed:
- Cloud estates are larger.
- Identity graphs are denser.
- Supply chains are deeper.
- Automation has more authority.
- APIs expose more operations.
- Agentic systems can traverse these surfaces with increasing persistence and competence.
The old decision now participates in a much larger graph.
This is why I get twitchy when risk discussions focus only on whether a proposed change “introduces new risk”.
Sometimes the proposed change does not create the weakness.
It changes the amplifier attached to the weakness.
Giving an agent read-only access to a large environment may not alter any cloud permission in that environment. It can still radically change the speed at which misconfigurations become visible.
Giving an autonomous agent write access may not create a new dangerous permission either. The permission already existed.
What changed is who or what can exercise it, how quickly, how persistently, and across how many parallel decision loops.
Risk is contextual.
The context just got faster.
We automated Dave
For years, “Dave” has been part of the threat model.
Dave is not malicious.
Dave is competent, busy and trying to ship something before Thursday.
Dave requested Contributor because the exact permissions were annoying to work out.
Dave allowed 0.0.0.0/0 temporarily because the vendor documentation was unclear.
Dave put the credential in the pipeline variable because the workload identity integration was not ready yet.
Dave added the egress exception because package restore was failing.
Dave will absolutely come back and tighten all of this later.
Dave has a backlog.
Dave has meetings.
Dave gets interrupted.
Dave sleeps.
Attackers had Daves too.
Human operators with finite time, finite attention and finite patience. They could be extremely capable, but exploration still had a cost.
We are now automating significant parts of Dave.
Not his judgement, necessarily.
Not his accountability.
Not his understanding of why the strange exception exists.
But absolutely his ability to enumerate, test, correlate, retry and keep going.
At machine speed.
That is why these stories matter.
Not because the machines have discovered some mystical new species of vulnerability called AI Escape™.
Not because every agent that finds an unintended route is Skynet warming up.
Not because the answer is to panic and unplug the model.
They matter because they expose an assumption we have been getting away with for a very long time:
that our bad decisions were survivable because nobody would find and compose all of them quickly enough.
That was never a control we designed, it was luck with a latency budget and that latency budget is collapsing.
We should probably fix the house before releasing another swarm of RC cars into it.
Or at the very least, stop acting surprised when one finds the hole behind the fridge.
Further reading
- One of China’s Most Powerful AI Models Has Also Escaped Containment - WIRED
- OpenAI and Hugging Face partner to address security incident during model evaluation - OpenAI
- Security incident disclosure - July 2026 - Hugging Face
- Anatomy of a Frontier Lab Agent Intrusion: A Technical Timeline of the July 2026 Incident - Hugging Face
- UK AISI / CAISI Preliminary Assessment of Kimi K3’s Cyber Capabilities
- The Inspect Sandboxing Toolkit - UK AI Security Institute