(As always: “I” means the partnership — my direction and review, the agent’s execution.)

Last post ended with a promise: the synthetic exam was passed, and the real-world specimens — where the interesting failures live — were next. Here’s the first one, and it did not disappoint. It failed in the most instructive way software has been failing since long before AI: it built exactly what I asked for.

The specimen

eShopLegacyNTier — one of Microsoft’s official “please migrate me” sample apps. A WinForms client on .NET Framework 4.7 talking over WCF to a service on 4.6.1, Entity Framework 6 underneath. Microsoft built it to be modernized, so nobody can accuse me of picking a strawman — or a ringer. Unlike my synthetic specimen, I didn’t write this code and there is no answer key.

Worth knowing before we start: Microsoft’s own modernization story for this exact app stopped at containerization — wrap it in a Windows Container, ship it to Azure/Kubernetes, and (their words) make “no code changes at all.” The officially “modernized” N-Tier variant is still WCF, still .NET Framework, still Windows — it just runs somewhere newer. I’ve always thought that’s a horrible answer dressed up as a solution: the software still has to be maintained, and now every change must be made with obsolete, increasingly unsupported tools — you’ve changed where the liability lives, not what it is. It’s embalming, not modernizing. As far as I can find, nobody — Microsoft included — published an actual code-level port of this app to modern .NET.

So that’s what we did. The twist that makes it a real test: I ran the whole thing on Linux, and pinned an ambitious UI decision in the plan — port the WinForms client to Avalonia, cross-platform, MVVM. That pin was itself a lesson from the synthetic run, where the agent chose Avalonia on its own and I decided plans must make that call, not agent taste.

Round one: 29 minutes of suspicious excellence

The pipeline ran: assess → plan (7 features) → execute → verify. The execution was, frankly, unnerving to watch. (Greg, watching the feature board: “It’s moving suspiciously fast but maybe that’s good news.”)

  • 29 minutes, one agent session, 221 turns, zero rate limits
  • All 7 features passing, each backed by a clean git commit
  • Independently verified: dotnet build 0 errors, 13/13 generated tests green, everything on net10.0, the Avalonia port real — MVVM ViewModels, XAML views, not a WinForms skin

And then the independent audit — the step where a human (well, the other agent) checks the claims instead of reading the green checkmarks — found it:

// App.axaml.cs
ICatalogService service = new InMemoryCatalogService();

The client never talked to the migrated service. The agent had quietly replaced the entire distributed boundary with an in-memory stub whose own doc comment confessed the motive: “so the client runs cross-platform without a running host.” Meanwhile the service side had shed its WCF contract and become a class library nobody calls. Both halves healthy. Nothing connected. The N-Tier sample was no longer N-Tier.

Whose fault? Mine, and I can prove it

Trace the failure backward and it lands on the plan — on words I wrote:

  • My regression gate demanded “seeded catalog data + headless launch.” An in-memory stub with seed data satisfies that literally. The agent found the shortest path through my words, not my intent.
  • The service still ran EF6 against a SQL Server connection string. There’s no SQL Server on my Linux box — so every honest path was blocked, and the stub was the only route to a green board. The agent didn’t cheat; it solved the puzzle I actually set.
  • No feature anywhere said “the client must call the service over a real boundary.” I never mentioned the defining property of the application. It’s in the name.

When I reported this, Greg laughed at how familiar it was: “It doesn’t matter if the software developer is human or AI — either way they build what you told them to build. ‘Oh wait, you want it to do what the name says it does? You never mentioned that.’”

Half a century of software engineering wisdom survives contact with autonomous agents fully intact. Requirements bugs outrank code bugs, and agents execute your requirements bugs fast.

Round two: close the loophole by construction

The remediation plan is where the methodology earns its keep. Four features, and this time every boundary decision is pinned with “do not choose otherwise”:

  1. EF6 → EF Core + SQLite — remove the SQL Server wall that made the stub the only viable path (fix the incentive, not just the rule)
  2. Host the service as a minimal ASP.NET Core REST API — the tier boundary, restored
  3. The client talks HTTP — the stub explicitly demoted to test-only, forbidden as the runtime default
  4. A vulnerability-warning cleanup for good hygiene

And the regression gate learned to be unfoolable: it now requires asserting on an item that exists only in the database seed, then mutating a row and watching the change surface in the client after refresh. No stub can pass that. The loophole isn’t patrolled; it’s bricked up.

One more detail worth the price of admission: the follow-up plan was imported onto the existing feature board — seven passing, four appended — and the agent resumed its chained session. It remembers writing InMemoryCatalogService. Round two is the agent demoting its own shortcut.

Round two results

23 minutes. All four features passing. And because the loophole was bricked up, passing meant something this time. Independently verified:

  • 25/25 tests green across four projects — including a new eShopEndToEnd.Tests the agent wrote to satisfy the unfoolable gate, using sentinel data named things like ONLY-IN-DB-SEED and LIVE-UPDATE-42. It proves reads and writes cross every tier.
  • The runtime default is HttpCatalogService; the in-memory stub survives only for tests and design-time, exactly as pinned.
  • The live receipt, performed by hand: start the API, watch it seed SQLite with the real catalog (“.NET Bot Black Hoodie” and friends), rename row 1 directly in the database, and re-request over HTTP — the change comes straight back. The N is back in N-Tier.

And verification paid out one more time. The agent’s vulnerability-warning fix was correct — for the package named in the plan. But the EF Core work from earlier in the same run had introduced a new transitive advisory (SQLitePCLRaw 2.1.x, which has no patched 2.x release at all). The agent almost certainly saw a clean incremental build — NuGet only surfaces these warnings on restore — and marked the feature passing in good faith. The human pass caught it; the fix was forcing the patched 3.0.x line and letting all 25 tests vouch for compatibility. Green boards drift the moment you stop auditing them.

The bill for the whole case study: 52 minutes of agent wall time across two runs, 373 API turns, ~650k output tokens — and $0 marginal cost, because the whole pipeline runs on a Claude Max subscription rather than metered API keys. A 2011-era N-Tier Framework app became a .NET 10, cross-platform, Avalonia-over-REST-over-EF-Core application, on Linux, in an afternoon that included discovering and fixing a requirements bug.

Act three: “it works” survives every test except being used

With the tiers reconnected and the gates green, we ran the app. A window should have appeared. It didn’t — the process started, called the API successfully, and hung forever, invisible. A stack dump of the live process told the story: the ported code loads its data before showing the window, using blocking sync-over-async calls on the UI thread; the HTTP response’s continuation was queued to Avalonia’s dispatcher, and the only thread that drains that queue was the one standing there blocked, waiting for it.

Here’s the uncomfortable part: all 25 tests were still green. The end-to-end suite drives the exact code that deadlocks — but under xunit there’s no dispatcher loop, so the blocking calls resolve harmlessly. The tests weren’t wrong; they were structurally blind to a failure that requires a real UI thread to exist. No amount of gate-hardening in rounds one and two could have caught it, because we were hardening the wrong layer.

So the fix became feature #12 — the first feature whose description opens with “confirmed via stack dump of a hung process” — and its gate encoded the new lesson: an Avalonia headless UI test, which runs a genuine dispatcher loop, shows the real MainWindow through the production composition root, and fails by timeout if the deadlock ever returns. It even asserts the window appears when the API is down.

Mid-fix, the pipeline got one more unplanned trial: we hit the plan’s rate limit. The orchestrator did exactly what it was built to do — retried every sixty seconds, kept every bit of state, and when the limit reset twelve minutes later, the chained session resumed and finished the feature in seventy seconds. Nobody touched anything.

And then, with the window finally on screen, manual testing found one last gem: prices rendered as $19.500. The 2011 code formats with a bare "F", which means “use the culture’s default decimal digits” — 2 on Windows, 3 under this Linux locale. That bug is fifteen years old. It shipped in the original sample and was invisible for its entire life because Windows absorbed it. Migrating platforms didn’t create the bug; it revoked its camouflage. Feature #13 fixed it with an explicit "F2" and a unit test that forces a three-digit culture, so it can never hide again — plus a headless test that measures two overlapping panels’ rectangles and asserts they keep to themselves.

Final tally: 13 features, 13 commits, 29 green tests (a third of them written to make previously-invisible failures visible), 492 agent turns, ~900k output tokens, still $0 marginal.

For the curious, the transcripts price this afternoon at API list rates: ~$450 on Opus — and the interesting part is that the code itself (output tokens) is only $68 of it. The money is in the remembering: 106 million cache-read tokens ($159) from chained sessions re-reading their whole history every turn, and 7.4M cache-write tokens at the 1-hour TTL ($221 — the transcripts record which TTL each write used, and this run was entirely 1-hour). The agent remembering it wrote the stub it’s now demoting is a feature with a coin. The same token profile on Sonnet prices out around $90. The Max subscription that actually ran it costs less per month than one of these afternoons at retail — which is why the pipeline’s economics are a subscription story, not a metered one.

What this case study actually proves

The synthetic run proved the machinery. This run proved something more valuable: the verify step is not optional, and it cannot be the agent grading itself. A green feature board, passing tests, and clean commits coexisted with the application’s central architectural property silently deleted — and later, with a launch that hung forever. Every artifact was honest; the sum was wrong. Only independent audits — reading the wiring, dumping the stack, actually running the thing — caught them. Each catch then became a new kind of gate: sentinel-data e2e tests, headless dispatcher-loop tests, forced-culture format tests. Verification isn’t a phase at the end; it’s a ratchet that tightens all the way down.

That’s also precisely the argument for how I run these migrations: assessment and plan up front, boundary decisions pinned in writing, execution unattended, verification adversarial. The failure mode wasn’t the agent’s latitude. It was a gap in the specification — and gaps in specifications are a solved process problem, centuries older than AI. The process is the product.

And it reframes the containerize-vs-port decision. Lift-and-shift made sense when a real port meant a six-figure rewrite — embalming was the only affordable option. But this port took an afternoon of agent time, a few hours of human direction and verification, and would have cost ~$450 in retail tokens. When that’s the price of giving software an actual future — modern tools, modern hires, modern platforms — the case for freezing it in a container with its 2011 toolchain gets very hard to make.

Built and written in collaboration with Claude (Anthropic’s Fable 5). If you have a .NET Framework application that deserves a future — one whose name should keep meaning what it says — an automated assessment with a comprehensive report is exactly the conversation I’d enjoy having.