Where agent supervision has to happen

Anthropic published Measuring AI agent autonomy in practice on 18 February, and there are a couple of numbers in it worth looking at. Newer Claude Code users run with full auto-approve in roughly 20 per cent of sessions, and that goes up to more than 40 per cent for experienced users. Interruptions go up as well, from about 5 per cent of turns for newer users to around 9 per cent for more experienced ones. It's one coding product and Anthropic lists several limitations, but I still think it says something useful about where supervision should sit.

Approving every command makes sense when an agent first gets access to a shell or a file system. It falls apart once a task needs dozens of reads, edits and test runs, because you start approving commands by their shape instead of their purpose. You can approve twenty of them and never notice the agent is solving the wrong problem. I'd keep individual approval for actions where the specific target carries the risk, like sending a message, changing production data, publishing a release, rotating a credential or running a destructive migration, and govern routine reads inside the workspace at a broader level.

What replaces the per-command prompt is a visible plan: the objective, the stages, which files or systems it expects to touch, how it'll verify the work and which actions will need another decision. Approving that plan gives the agent a bounded envelope. For example, it can read the repository, edit one application package and run local tests, but network access, deployment config and writes to external services stay outside. When the plan changes in a way that matters, show the difference. "One test failed" is just normal feedback. "The fix now needs a change to the shared authentication library" is a new scope decision, and the interface should pause there, explain why and let you accept, redirect or stop.

Interrupting only helps if you can see enough to judge the run. A spinner and an elapsed time won't do it. I'd want the current stage, the last meaningful action, what's changed and whether verification passed, plus an alert when the agent goes outside scope, repeats a failed action or asks for broader permissions. The stop button also needs to be honest about what it does. Cancelling future tool calls, killing a local process, rolling back uncommitted work and asking the model to stop after its current action are all different, and you should know which one you've just pressed.

After a state-changing action, I'd also check the result through a separate route where that's practical. A successful tool response only tells you the tool accepted the request. Read the target back (for a deployment, the released revision and health signal), and if it doesn't match, stop anything that depends on it and show the mismatch.

That's where I think the Anthropic numbers point: experienced users approving less command by command and interrupting more fits supervision that sits on an approved plan and on the few actions where the target carries the risk, with enough visibility in between to step in. To test a setup like this, I'd try one long low-risk task and one short consequential action. The first should run without constant prompts and still be easy to redirect, and the second should stop exactly where the target and effect are known, then show what happened afterwards. If both get the same approval pattern, the supervision is probably still attached to commands and not to what those commands do.