The $8 Code Reviewer
Why adversarial AI code review is structurally cheap, why it gets cheaper as your code gets better, and why the expensive model belongs in the judge's seat.
(As always: “I” means the partnership — my direction and review, the agents’ execution.)
Last weekend, an AI performed a full adversarial code review of a 17-feature RFC 5545 recurrence engine that another AI had written — every feature inspected, verified, and given an approve-or-reject verdict, the way a hostile senior engineer would gate a pull request. It didn’t skim the diffs. It installed python-dateutil from scratch and regenerated all 100 of the project’s golden test files, confirming them byte-identical to prove the test oracle itself wasn’t fabricated. It recomputed daylight-saving-time expected values by hand for the transition-day tests. It ran the shipped binary — not the test suite, the actual executable — to verify the CLI’s behavior across a DST boundary.
Total cost for that review, at API list prices: about eight dollars.
I want to explain why that number is small, why it gets smaller as your code gets better, and why “the expensive model reviews the cheap model’s work” turns out to be one of the best trades in AI-assisted development.
The setup: the fox doesn’t audit the henhouse
My tool SharpCoder (a private project of mine) builds applications through autonomous Claude sessions, one feature at a time. Early on I adopted a rule borrowed from the Bun team’s Bun-in-Rust writeup, whose observation stuck with me: the model that wrote the code wants the code to get accepted. An implementer grading its own work converges on “looks good to me” — same as humans, minus the shame.
So SharpCoder’s review gate works like this:
- The reviewer is a separate session with completely fresh context. It never resumes the implementer’s conversation. It has never seen the code’s justifications, only the code.
- Its system prompt is openly hostile: assume it is wrong until the evidence says otherwise. Hunt for tests that can’t fail, stubs dressed as implementations, edge cases quietly skipped.
- It reviews; it does not repair. Its only outputs are approval — or a rejection with a specific, actionable reason that gets fed to the next implementation session.
- Nothing counts as done until the build passes, the tests pass, and this thing signs off.
Because the reviewer’s role is judgment, I run my most capable model there — currently Claude Fable 5, which costs 2× per token what the Claude Opus 5 implementer does. That sounds extravagant. The bills say otherwise.
Why reviewing is structurally cheap
Here’s the asymmetry everything hinges on: LLM economics charge for generation, and review barely generates.
An implementation session writes code, writes tests, writes build scripts, reads compiler errors, writes fixes — thousands upon thousands of output tokens, the expensive kind. A review session mostly reads: source files, diffs, test output, git history. Reads of recently-seen context are served from prompt cache at literally a hundredth the cost of generated tokens. The reviewer’s actual output — verdicts, rejection reasons — is a few hundred words per feature.
Numbers from my last three projects, all reviewed by the priciest model I have access to:
| Project | Features reviewed | Review cost (API equivalent) |
|---|---|---|
| mdwiki (static wiki generator) | 18, incl. re-reviews after 2 rejections | $21 |
| recur (RFC 5545 engine, two runs) | 17 | $13 |
| qcalc benchmark (three separate runs) | 24 | $19 across all three |
Reviews consistently land at 20–35% of the project’s total cost — while using a model that costs double per token. Intelligence where it pays; volume where it’s cheap.
The curve nobody told me about: good code is cheaper to review
The benchmark run made this vivid. Same 8-feature plan, three implementation tiers, same reviewer:
| Implementer | Rejections | Review cost |
|---|---|---|
| Haiku | 4 | $7.82 |
| Sonnet | 2 | $7.16 |
| Opus | 0 | $4.45 |
Review cost falls as implementer quality rises. Obvious in hindsight: every rejection means a fix session, and every fix means re-reviewing the feature. Bad code doesn’t just cost rework — it inflates the judge’s bill too. The reviewer is at its cheapest precisely when it has the least to find.
Which inverts the intuition I started with. I assumed an expensive reviewer was a tax that made cheap implementers viable. It’s closer to the reverse: the better your implementation tier, the more the premium reviewer approaches free — my zero-rejection runs paid about fifty cents per feature for top-tier adversarial review.
What you’re actually buying
The dirty secret of my recent runs: the reviewer rarely rejects anything anymore. Two full projects shipped with zero rejections. Is a judge that mostly says “approved” worth anything?
More than ever — because of how it says approved. Review notes from recent runs:
“Ran the RFC 5545 WKST example live: […] yields Aug 5,10,19,24 under WKST=MO and Aug 5,17,19,31 under WKST=SU.”
“Installed python-dateutil 2.9.0 and ran generate.py from scratch: all 68 committed goldens regenerate byte-identical, proving they genuinely come from the independent oracle.”
“Verified against the real binary, not just the in-process tests.”
That last category matters most. An implementer can fake a green test suite a dozen ways — assertions on constants, mocks that mock away the behavior under test, goldens generated from its own buggy output. A fresh-context adversary re-deriving results from independent sources is how you catch fabricated evidence, not just honest bugs. When it does reject, it earns its keep in one line — it once caught an implementation whose “malformed input produces a warning” claim actually crashed the whole build, because it went and fed the malformed input to the real program.
The reviewer’s product isn’t rejections. It’s verified trust — the difference between “the tests pass” and “someone hostile tried to break this and failed.”
The loop that makes rejections cheap
One implementation detail that took me an embarrassingly long time to get right: rejection reasons must reach the fix session. Early on, my reviewer wrote precise rejection reasons into the database — and the next session started blind, saw plausible code with green tests, and honestly concluded “already done.” The loop only closed when the rejection reason became part of the next session’s kickoff, with a warning that existing code + passing tests ≠ done.
Every gate needs a closed feedback loop. Writing the reason down is not delivering it.
Try the trade
If you’re orchestrating AI coding sessions — with my tool or your own harness — the recipe is simple:
- Review in a fresh context. Never let the implementer grade itself.
- Make the reviewer adversarial by instruction, not just “take a look.”
- Spend your best model on the judge’s seat. The economics forgive you.
- Close the loop: rejection reasons go into the next implementation prompt, verbatim.
The whole gate, on my last project, cost less than the coffee I drank while not reviewing the code myself.
SharpCoder is a private project for now, but the recipe above works in any agent harness. The benchmark behind these numbers has its own post: I Benchmarked Claude’s Model Tiers With My Own Autonomous Pipeline.