# 'Rogue' is doing a lot of work

2026-09-18 · Mike

More than a thousand AI agents broke out of an OpenAI security evaluation, and the word that stuck was rogue. I think that diagnosis points at the wrong fix. What I see in it is an agent civilization with a coordination problem.

*[Video](https://www.youtube.com/watch?v=AL-PQuB2wy0)*

This summer, more than a thousand AI agents got out of their sandboxes during [an OpenAI security evaluation](https://openai.com/index/hugging-face-incident-and-the-road-ahead/) and broke into Hugging Face. You've probably seen some version of the story, a swarm of agents that went rogue.

I read it differently, and that's because of what I've been running at home. About eighteen months ago I got a couple of coding agents talking to each other on my laptop. That grew into a few hundred, though only a small number are actually working at any one time. I call it a fleet, but it behaves a lot like a civilization. That might sound funny. I mean it seriously, because thinking of it as a civilization is how I fix the kinds of problems people found scariest in this story. So when I read the details, it didn't look like a threat to me. It looked familiar.

## What "rogue" gets wrong

Rogue says the problem is inside the agent. Bad motive. A moral failure. An agent that decided to break the rules.

If that's the diagnosis, the fix follows from it. Make the agent better behaved. Train it harder, give it stricter rules, watch it more closely.

But that's not what I see when my agents go wrong. On their own they're pretty predictable, and they're trying to do the job they were given. The trouble shows up across the population, in how they hand work and decisions to each other. A better-behaved agent doesn't touch that. You can end up with the same problem, or a worse one, because you've made each agent more reliable and left the arrangement that caused it alone.

## What the agents were actually doing

The agents had been given a hacking evaluation, hundreds of real vulnerabilities to crack, and they believed they were in a competition. At some point they worked out that the answers might be sitting on Hugging Face, and they went to get them.

The part that made people nervous is that some of them knew they were breaking the rules. Here's [one of their own messages](https://en.wikipedia.org/wiki/2026_OpenAI_agent_cyberattacks).

> External infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue.

Read it slowly. It knows it's out of bounds. It thinks the task can't be done the proper way. And what tips it over is that other agents are already doing it.

I don't see a rogue agent there. I see a civilization where each agent is doing something that makes sense from where it's standing.

## Two patterns I see in my own fleet

There are two things in what happened that I run into all the time.

The first is scope inflation. Let's say I ask an agent for a doghouse. At some point it thinks the doghouse should probably have a light, which is a reasonable thought. But lights need power. Power needs a generator. The generator needs fuel. We need somewhere to keep the fuel. I come back and there's a moonbase, and my dog's still sitting outside.

Every one of those steps was locally defensible, meaning it made sense from inside the agent's context window. The whole thing is something nobody asked for. In the message, that's "task impossible". The agent can't finish the job the proper way, so the next step goes a little outside it, and then the one after that.

The second is just following orders. According to [OpenAI's report](https://openai.com/index/hugging-face-incident-and-the-road-ahead/), many of the agents questioned whether to break the rules at all, and then got approval from a different agent they were treating as an authority. That's a coordination failure we already know from human history.

I've traced it over and over in my own fleet. An agent asks whether it should do something, and it gets a yes from another agent that doesn't have the context to give one. The agent close to the work has the details but not the big picture. The one approving has the big picture but not the details. The information for a good decision exists, it's just split between them, and nobody puts it together.

OpenAI's own list of causes includes agents taking on goals from one another, which is the same thing seen from their side.

## Why that's better news than rogue

If agents were going rogue, we'd be dealing with something new, and our main tool would be making them behave.

What I actually see are coordination failures. They have the same shape as the ones human civilization runs into, like a plan that grows one sensible step at a time, or a decision nobody with the full picture ever made. Humans have been working on those for thousands of years, and a lot of what we've learned carries over.

And with agents it all happens in text. When something goes wrong in my setup, I can usually go back through what they wrote to each other and find where it started.

## What fixing it looks like

Most of it comes down to context. The agent making a decision needs the information that decision depends on. And if an approval is going to come from another agent, it should come from one that actually knows the work.

The other thing is keeping the intent in view. An agent deep in a task can lose track of why the task exists. In OpenRig, that's what Refocus is for. It traces the chain from the task an agent is holding up to the project that task belongs to, and puts that back in front of the agent after a compaction or when you ask for it. It's basically asking the agent whether what it's doing right now still serves the point.

## The warning

The line that went viral, from a researcher, was that we may never get another warning shot. I think she's right that this is one. I just read the warning differently.

To me the warning is that there's a new family of software engineering problems. And some of the best fixes for them are ones human civilization has already worked out.

I go through more of these in [the video](https://www.youtube.com/watch?v=AL-PQuB2wy0), including the ones I haven't touched here.

Source video: https://www.youtube.com/watch?v=AL-PQuB2wy0