(As always: “I” means the partnership — my direction and review, the agent’s execution.)

Post one on this blog reported a falsified hypothesis: I built semantic code tools for my AI agent, recommended them in its prompt, and it ignored them — completing a navigation-heavy task with Read and grep while my tools sat connected and idle. The diagnosis then: advisory prompting produces token compliance, not adoption, and tools that merely do what grep does slower-but-better don’t change behavior.

So we built v2: tools that do what grep cannot do at all. A solution-wide semantic rename that catches overrides and call sites while refusing to touch string literals. Interface extraction. Dead-code detection with honest confidence tiers. (Built, tested against adversarial fixtures, shipped — by the same agent pipeline this blog keeps writing about, in 43 minutes, but that’s not today’s story.)

Then round 2 of the experiment: a fresh two-project solution the agent had never seen, and a rename task strewn with traps — a string literal spelled exactly like the method, a ValidateHelper class, a lowercase local, prose comments, a derived override. Task wording never mentions the tools. Three arms.

Arm A: same advisory guidance, new capabilities. Adoption: zero.

With the identical “prefer semantic tools” guidance from round 1 — and apply_rename now sitting right there in the tool list — the agent made zero tool calls. Not even a solution load this time. It hand-edited three files.

And here’s the uncomfortable part: it got everything right. Every trap survived. Careful targeted edits, not blind search-and-replace — it even updated the prose comments to track the new name, which the semantic rename deliberately doesn’t do. At three files, a strong model doesn’t need your tool, knows it, and acts accordingly. Capability alone changes nothing.

Arm B: directive guidance (“you MUST use apply_rename”). Adoption: nine attempts, zero successes.

Change the prompt from prefer to must, and behavior changed instantly — the agent went straight for the server. Then something I didn’t expect: five consecutive load_solution calls, each dying in under a second, each answered with the SDK’s opaque "An error occurred invoking 'load_solution'." After nine attempts at the locked door, the agent quietly did the rename by hand. (Correctly, again.)

The root cause is the kind of bug you only find by watching a real caller: my tool’s parameter is named path; the model guessed solutionPath — a perfectly natural guess for a tool named load_solution — and the framework’s binding failure gave it nothing to self-correct from. Five identical guesses, five identical stone walls.

My agent wasn’t refusing my tool. My tool was refusing my agent. And it did so silently: a green task result, a correct rename, and a user who would never know the expensive semantic machinery contributed nothing.

Arm B2: same directive, one ergonomic fix. Adoption: complete.

The fix took minutes: accept both parameter spellings, and when neither is given, return a structured message that says exactly what to pass. Rerun. The agent loaded the solution, searched for the symbol, ran a preview of the rename — twice, unprompted, checking its blast radius before committing — then called apply_rename. Zero hand edits. Every trap survived. Build clean, output identical.

What I now believe about agents and tools

  1. Advisory prompting is decoration. Two experiments, two task types, zero adoption. If you need an agent to use a tool, say must, not prefer.
  2. Tool signatures are UX for a language model. Name parameters what a model would guess. Accept aliases. And never, ever let a binding error return a generic message — an agent that can’t diagnose a failure doesn’t retry harder, it silently routes around your tool forever.
  3. Watch real transcripts. Every finding in this post came from reading what the agent actually did, not what I assumed it did. The failure was invisible from the outcome — the rename was correct either way.
  4. The correctness case for semantic tools lives at scale — and so, it turns out, does adoption. We ran the scale round the same afternoon: 300 call sites of the target method interleaved with 200 same-named-but-unrelated decoys across 100 files, machine-scored against a generated answer key, with predictions registered and committed before the first run. My predictions were wrong, and I’m delighted to report it: at that scale the agent adopted the semantic tools with zero guidance at all — no advisory, no directive, nothing — and scored perfectly. Four arms (including Sonnet), four perfect scores, all through apply_rename; not one arm was willing to hand-edit 300 sites. Adoption was never a prompting problem. It’s a break-even problem: below the threshold where text tools visibly fail, no prose moves an agent; above it, silence is enough. Your job is to make sure that when the agent arrives at your tool — desperate, unprompted, guessing parameter names — the door opens.

Built and written in collaboration with Claude (Anthropic’s Fable 5). The tools — and both experiment writeups — are public: github.com/sharpdaddy59/RoslynMcp. If you have a .NET Framework application that deserves a future, an automated assessment with a comprehensive report is exactly the conversation I’d enjoy having.