(As always: “I” means the partnership — my direction and review, the agents’ execution.)

Last time, one appended sentence — “the enumerated cases are a floor, not a boundary” — bought back all the curiosity that explicit acceptance criteria had suppressed, and then some. Sonnet went 96 → 171 tests, Opus 387 → 613. I ended with a spec recipe: name every landmine, then mandate exploration beyond the list.

But that recipe has a smell. The mandate was pasted into every feature’s acceptance criteria — eight features, eight copies of the same sentence. If it works, shouldn’t it work said once, in the system prompt, where a tool like SharpCoder could ship it as an invisible default? That’s bench4: same 8-feature calculator plan, same explicit landmine-naming criteria, same fixed reviewer (Claude Fable 5), and the exploration clause moved into a per-project system-prompt override. Haiku sat this one out — two DNFs earned it a bench pass.

One deliberate asymmetry: the reviewer runs on its own prompt and never sees the mandate. In bench3 the clause lived in the acceptance criteria the reviewer was checking, so it could enforce exploration. Here, only the implementer hears it. That isolates the question I actually care about: does the sentence change what gets written, or just what gets policed?

The predictions were registered in the script header on July 30th; the run sat staged for three weeks (life, and a dead internet connection, intervened) and fired the day my quota was about to expire anyway. Predicted: partial transfer — Sonnet 120–150 tests, Opus 450–550. Ship-as-default gate: ≥160 and ≥580. Dead-letter gate: ≤110 and ≤410.

Tests written / rejections

  Explicit (no mandate) Explicit + explore in ACs Explicit + explore in system prompt
Sonnet 5 96 / 2 171 / 1 125 / 0
Opus 5 387 / 0 613 / 0 454 / 0

Finding 1: Partial transfer — both arms inside the predicted bands

Sonnet 125, Opus 454. Both landed in the registered windows, both near the bottom edge. Moving the sentence from the acceptance criteria to the system prompt kept roughly a third of its effect — Sonnet recovered 29 of bench3’s 75-test gain, Opus 67 of 226.

I find the shape of that result more interesting than the number. The words are identical; only the distance from the work changed. A clause inside the acceptance criteria sits in the model’s hands while it’s satisfying them. The same clause in the system prompt is ambient — culture, not checklist — and ambient instruction decays over a long agentic session the way a stand-up announcement decays by Thursday. Two benches ago I concluded that explicit specs “arm the reviewer.” The generalization now looks like: proximity is potency. Where you say it is part of what you said.

Finding 2: The one I didn’t predict — friction collapsed

The registered predictions were about test counts. The headline turned out to be everything else:

  • Zero rejections, both arms. Under identical reviewer conditions, bench2’s Sonnet took 2 rejections. The system-prompt mandate produced first-pass work the reviewer couldn’t refuse — even though the reviewer had no idea the mandate existed.
  • Two sessions per arm, total. One coding session that carried all eight features to green, one review session that approved all eight. Twenty-six minutes wall-clock for the whole Sonnet arm.
  • The cheapest Sonnet run of the entire series: $11.21 all-in, $1.40 per approved feature (vs $2.26–$2.43 in every earlier bench). The review bill was $3.62 — half of bench1’s. The review-cost-tracks-implementer-quality curve from the first post keeps paying out.

So the clean claim bench4 buys: the exploration mandate improves what the implementer writes even when nobody is checking for it. In bench3 I couldn’t distinguish implementer effect from reviewer enforcement. Now I can: enforcement is where the test-count multiplier lives, but the first-pass-quality gain is implementer-side and survives the move.

What ships in SharpCoder

Both placements, because they compose. The mandate goes into the default system prompt — an invisible floor that costs nothing and collapses review friction. And SharpUpgrade keeps emitting it into acceptance criteria, where proximity and reviewer enforcement buy the full thoroughness effect. The system prompt sets the culture; the acceptance criteria make it law.

The usual honesty

n=1 per arm, one domain, one reviewer, as always. This was also the first bench on a new prompt-delivery path (ClaudeCode.Net 0.2.5 pipes long prompts over stdin) — I can’t fully rule out that as a contributor to the friction collapse, though it has no plausible mechanism for raising test counts. And the two-session pattern means the per-feature cost figures ride on one long session’s economics rather than eight short ones.

Scorecard on the registered predictions: two for two, both inside the bands — the first bench in this series where nothing I predicted was wrong. Naturally, the most important result was the one I didn’t think to predict.

Previous entries: the tier benchmark, I Was Wrong About Specs, and The Escape Hatch.