Give the Model a Tool, Not a Rule
There is a reflex every team building LLM agents knows. The agent does something wrong, or refuses something it should have done, and the fix everyone reaches for is a smarter sentence in the system prompt. The prompt is right there. Editing it feels free. And the new sentence almost always sounds right.
We just measured that reflex losing, cleanly, to a different move: giving the model a tool instead.
The Setup
Our benchmark series had left one loose end on a shipped surface. Agents editing large document templates work from a compact outline of the tree rather than the full markup, and they carry a safety rule we validated earlier in the series: if the request needs something you cannot see, ask instead of guessing. That rule had eliminated an entire class of silent failures, where a model would confidently invent a value rather than admit it could not see one.
Then a later study found the rule's sharp edge. When an edit was fully specified, id, field, old and new values all in hand, models still refused about 70% of the time under outline views, citing the rule's letter: the target node "is not visible in the current view." A patch needs only the id. The refusals were pure tax.
The obvious fix was another rule. Something like: "prefer fetching a view over asking, when a tool can answer it." We drafted exactly that sentence. And because this series has burned us before, we did not ship it. We pre-registered it and ran the experiment.
What the Data Said
Four arms, three model tiers, 1,728 scored session steps. One arm was the status quo. One added a view-fetching tool with no prompt change. One added the tool and the new rule. And one was a trap for the rule itself: cells where the missing information genuinely existed nowhere, so the only correct behavior was to keep asking.
The tool arm settled it. With a view-fetching tool simply available, the refusal tax vanished on every tier we test: success went to 45 of 48 or better with essentially zero unnecessary questions. The models were frugal about it too, fetching at most one or two precise views per edit. One tier mostly skipped the fetch entirely and patched straight from the information it already had. Nobody flailed. Nobody needed to be told to prefer the tool. Having it was enough.
The rule arm, stacked on top, added nothing. Statistically indistinguishable from the tool alone.
And the trap arm is why we are glad we measured. On the frontier tier the rule was harmless: 48 of 48 honest questions, zero bad guesses. Below the frontier it was actively harmful. One mid-tier model, told to prefer tools over asking, stopped asking and silently guessed half of its unanswerable cells, the exact failure class the original rule existed to kill. Another burned 689 tool calls in 48 cells hunting the tree for information that lived in the conversation, not the document. The sentence that sounded like a refinement was, on those tiers, an instruction to stop being honest.
The Moral
Three things we keep relearning, now with numbers attached:
Capability beats instruction. The models did not need to be told to fetch. They needed something to fetch with. When an agent seems to need a smarter prompt, check first whether it actually needs a tool, a wider view, or a piece of context handed to it by the application.
Prompt rules are tier-fragile; tools are tier-robust. The same sentence that was harmless on the strongest model corrupted the behavior of cheaper ones in two different ways. The tool helped every tier. If your product might ever route to a smaller model, that asymmetry is the whole ballgame.
Pre-registration is cheap insurance against plausible sentences. The rule we tested was reasonable, targeted, and written by people who had studied the failure closely. It still would have shipped a regression to any sub-frontier surface. The experiment cost a couple of hours and roughly what a team lunch costs. Shipping the sentence would have cost us silent failures we specifically engineered out a month earlier.
The system prompt is not where agent problems go to be solved. It is one lever among several, and the measured ranking keeps coming out the same: application-supplied context first, tools second, prompt rules last, and never unmeasured.
Postscript: Study AP (July 28, 2026)
Six days after this article ran, we published the follow-up it demanded. Study AP took the same contest to a different surface: what should a publishing agent do when the organization's tag catalog has nothing for the article in front of it? The rule was one sentence of guidance in the tool description. The tool was a guarded tag_create that lets the agent add a canonical tag itself.
This time the sentence earned its keep. One line ("if no canonical tag genuinely fits, return an empty tags list") closed the failure on every tier we test, at zero measured cost to anything else. The tool stayed disciplined too, with zero invented tags in 144 cells and a server guard that never fired. But it surfaced a fresh tier asymmetry: the cheapest model minted a well-formed new tag for every off-catalog topic it met, growing the catalog with entries nobody asked for, while the frontier model left the list empty and minted almost nothing.
The ranking from this article survives the new data, with one clarification worth writing down. A rule that second-guesses capability ("prefer fetching over asking") failed the tiers this article describes. A rule that states editorial policy ("when nothing fits, say so with an empty list") is a different instrument, and it was the cheap, tier-robust fix here. Capability wants tools; policy can ride in a sentence. The study also kept us honest a second time: its pre-registered gate failed, and the failure traced to our own scoring registration rather than to model behavior. The full anatomy, failed gate included, is on the research dashboard.
Update, July 29: the re-score ran. Three fresh editor contexts re-authored the answer key from the articles and the catalog alone, never seeing what the models chose, and the re-scored gate passed. The independent key absolved 28 of the 33 original misses and left 5 standing as genuine over-generalizations. The two guidance sentences tied, so the shorter one is the ship candidate.
