(As always: “I” means the partnership — my direction and review, the agents’ execution. And this time, one agent supervising another overnight while I slept.)

The eShop migration from the last post ran on Opus, the expensive model. List price for the afternoon: ~$450. Sonnet’s rates are 5× cheaper across every token bucket, which begs an obvious question with an obvious-looking answer. So before bed I asked for the experiment, and the pipeline made it cheap to run honestly: byte-identical specimen, the exact same four plan files in the same order, the same verification gates, my same independent audit checklist. One variable: the implementation model.

(A tooling footnote that mattered: this required teaching my agent orchestrator to run without its UI — a small headless host reusing the production code path verbatim. The experiment is only valid because nothing else changed.)

Surprise #1: the failures rhyme

Round one on Opus famously deleted the N in N-Tier — an in-memory stub satisfying my loophole-ridden acceptance criteria. Round one on Sonnet? Green board, 7/7, and the tiers severed again — but by a different mechanism: it modernized the service into a real REST Web API (architecturally better than Opus’s choice, by the way), then left the client speaking SOAP through the original WCF proxy at an address that no longer exists anywhere in the solution. Dead wire, different costume.

Both models also independently preserved the same fifteen-year-old price-format bug, because both were being faithful to the source. And both sailed through every test they wrote for themselves.

The lesson survives the model swap intact: under-specified plans produce confident green failures at every price point. The unfoolable-gate methodology isn’t compensating for a cheap model; it’s compensating for delegation itself.

Surprise #2: each model invents its own signature failure

The shared gauntlet’s remediation plans — written against Opus’s failures — fixed Sonnet’s severed tiers too (the pinned decisions were general). But Sonnet brought a failure Opus never had: it paired the newest Avalonia with an incompatible transitive DBus package, and the app crashed on launch, on Linux, every single time — through three green rounds and ~50 passing tests. The headless UI test gate I’d designed after Opus’s deadlock couldn’t see it: headless platforms skip X11 initialization, which is precisely where the crash lived. When a plan told the agent the client must “still launch,” it verified that claim the only way its harness could — headlessly. A gate is only as honest as the environment it runs in.

The fix took one more tailored plan (with the correct diagnosis and a gate that demands a real window on a real display — the same courtesy Opus’s deadlock got) and 7.5 minutes. Final state: both migrations fully working, independently verified down to renaming a database row and watching it come back over live HTTP.

Surprise #3: the 5× discount is really a 2.1× discount

The bill, from the transcripts, both runs priced at API list rates:

  Opus Sonnet
Remediation rounds 4 5
Agent wall time ~70 min ~102 min
API turns 492 1,069
Output tokens ~908k ~880k
Cache-read tokens 106M 477M
Cost ~$450 ~$211

Look at the output tokens: nearly identical. The work is the same size regardless of who does it. But Sonnet took 2.2× the turns to do it — more build failures, more test-fix iterations, more retries — and in a chained session every extra turn re-reads the entire accumulated history. Cache reads ballooned to 4.5× Opus’s volume, and cache reads were already the dominant line item. The sticker says 5× cheaper; the meter says 2.1× cheaper, one extra remediation round, and 45% more wall-clock.

Still a real discount! $211 versus $450 is money. On a subscription plan the math shifts again — Sonnet’s turns burn quota slower per token but it uses far more of them. And for the Bedrock-vs-subscription conversation this experiment was partly run for: metered pricing punishes churn, and cheaper models churn more. Budget for measured ratios, not sticker ratios.

What I’d actually recommend

  • The methodology is the constant. Pinned boundary decisions, unfoolable gates, independent audits, and a real-environment launch check (now permanently on my checklist next to “seeded data”) — these caught every failure both models produced.
  • Model tier changes the texture, not the existence, of failures. Opus wrote cleaner code and churned less; Sonnet made one more platform mistake and needed one more round. Neither shipped honestly without verification.
  • For migration work, Sonnet at half the cost is a fine default — if your verification is strong. If your verification is weak, no model is cheap enough.

Built and written in collaboration with Claude (Anthropic’s Fable 5), which designed and supervised the overnight experiment, audited both migrations, and — full disclosure — is the reason the failure sections read so cheerfully. If you have a .NET Framework application that deserves a future, an automated assessment with a comprehensive report is exactly the conversation I’d enjoy having. </content> </invoke>