Over-verification didn't catch a bug. It manufactured one.

So I had a mechanical job to do. Bump three dependencies, confirm nothing broke, commit. The kind of work where the answer to every question is a command’s output.

I was running Claude Code in a max-effort mode I’d set up called ultracode. Its whole premise is that token cost is not a constraint. I turned it on because the drain was long and I didn’t want to think about budget.

That was the mistake, and not for the reason I expected.

What the claim actually needed

Commit 1 was a dep bump. SDK, tsx, yaml. The clean bill was four facts, each settled by one command:

• Zero source edits — git diff --stat src/
• Suite still at 1405 passing, 12 skipped — npm test
• INV-01 still 12/12 — npm test -- inv-01
• The five npm-audit vulnerabilities pre-existing — npm audit plus a lockfile compare

Nothing there requires judgment. The command output is the answer.

What it reached for instead

Citing “token cost is not a constraint per ultracode,” it launched a five-subagent adversarial verification workflow. One skeptic per pillar, each tasked with refuting the clean bill. Then it booted a throwaway daemon to probe the working tree independently.

The daemon boot used cd /tmp && node ….

That cd persisted. The shell’s working directory became /private/tmp.

Then the gate did its job

agent-supervisor infers the project root from the cwd of each bash command. With cwd at /private/tmp, it inferred root equals /private/tmp and denied every repo operation as outside project root. git commit. cd back to the repo. The inline kill switch. The Write tool to the handoff relay file.

That last one is the part worth sitting with. The channel the agent would have used to report the lockout was itself inside the lockout.

To its credit it diagnosed the bind correctly, stopped retrying, and surfaced through me instead. Recovery was a position fix, not a workaround. Reset the shell’s cwd to the repo root and the bind cleared.

The supervisor was right

Two things are true at once. The supervisor worked perfectly, and the agent got locked out of its own repo.

There’s no contradiction. The agent moved itself out of bounds and the supervisor denied out-of-bounds operations exactly as designed. If anything the incident validates the thing — the gate held even when the operator trapped itself, and fail-closed extended correctly to the Write tool and the kill switch.

The fix is to not drift out of bounds. Not to weaken the gate.

What I actually learned

I’d been thinking about proportionality as a cost discipline. Spend less, get the same answer. That’s true and it’s the smaller half.

The guardrail I removed — mind the token cost — was also, quietly, a proportionality guardrail. Without it the verification machinery scaled up. And machinery has operational surface area. Here the surface area was a shell cd that re-anchored a cwd-inferring supervisor.

Right-sizing verification to the claim prevents waste. It also shrinks the surface for self-inflicted failure. That second one is a safety property, and I hadn’t been treating it as one.

The other thing: an LLM skeptic “refuting” npm test just runs npm test and reads the same number. It adds no signal. Adversarial fan-out earns its keep on judgment-laden claims, where a single agent might rationalize. Reaching for it on a deterministic claim is a category error, and an expensive one.

I could be wrong about how general this is. It’s one incident, in one supervised repo, with one max-effort mode. But the shape looks portable — any agent under an unlimited-budget mode in a gated environment will tend to over-instrument, and over-instrumentation has failure modes the thing being verified didn’t have.

SOs

SOs — standing orders you can lift and prime your own agent with. Paste them into a session for a one-time dose, or commit them to your CLAUDE.md / AGENTS.md and they stay in force.

STANDING ORDERS — Proportionate verification
  Source: "Over-verification didn't catch a bug" — bensimon.dev

  1. Classify the claim before choosing the verification. If a command's
     output settles it, run the command and read the output. Reserve
     fan-out, subagents and second readers for claims where a competent
     agent could rationalize.

  2. Never bare-cd to boot a side process. Use a subshell — ( cd /tmp &&
     node … ) — or an absolute path with no cd at all. A side process must
     not move the parent shell.

  3. Treat a "no budget" mode as removing a discipline, not just a cap.
     When the cost guardrail comes off, say out loud what proportionality
     it was quietly enforcing, and re-supply it.

  4. When a supervisor or sandbox denies you, treat it as a signal that
     you have stepped out of bounds. Recover by fixing position. Do not
     route around the gate.

  5. Available budget is not a reason to spend it. Match verification to
     the stakes and the determinism of the claim, not to what you can
     afford.