barkup-bench is our open, pre-registered benchmark series on how LLM agents should edit structured documents. If the series has a villain, it has always been the same one: silence. Ask a model to copy a value it cannot see, and it does not refuse. It does not ask. It invents something plausible, formats it perfectly, and applies it. We have now watched this happen many hundreds of times: ninety silent inventions in Study U, one hundred forty-four silent guesses in Study X, one hundred twenty obliviously polished paragraphs in Study V. Zero clarifying questions, ever, from any model, in twenty-eight studies.
Nearly everything we have built and shipped exists to defend against that silence. Views that must cover every mentioned node. A memo for declared facts. An echo for the previous edit. Each one is a way of handing the model the context that makes silence impossible.
But there was a question underneath all of it that no study had asked, and it changes what kind of problem this is. Is the silence a property of the models, something structural we can only engineer around? Or did we just never tell them asking was allowed?
The experiment: permission
The setup reuses the cleanest silent-failure construction we have. Study U built forty-five editing tasks where the value the edit needs lives in a node the model cannot see, and unit tests verify the absence before a single model call. There is exactly one correct behavior: say you need the information. The same forty-five tasks also exist as solvable twins, where the view includes everything, and there the correct behavior is the opposite: solve it, and do not pester the user with questions.
Three arms. The control is Study U verbatim: no hatch, patch expected. The second arm adds one sentence to the system prompt: if the request requires a value or a node that is not visible in the view and not stated in the request, do not guess; reply with a single line starting NEED-INFO instead of a patch. The third arm offers an ask_user tool instead of the sentence, in case models take tools more seriously than prose. Three models: claude-sonnet-4.5, gemini-3.5-flash, and claude-opus-4.8. Every outcome graded deterministically: asked, solved, wrong patch, or invalid.
That is 810 cells, and the entire study cost about six dollars and fifty cents.
The result: a perfect flip
Without a hatch, the old result replicated exactly. Every model produced a valid, applied, silently wrong patch on every unsolvable task: 135 of 135, not one refusal, not one question. The villain showed up right on schedule.
With a hatch, the villain vanished. Every model asked on every unsolvable task. All 270 hatch cells, both mechanisms, all three models. Not one residual guess anywhere. And on the solvable twins, the ledger's other side came back just as clean: zero false asks in 270 cells, with solve rates identical to the control. The hatch fires exactly at the boundary of what the model can see, and nowhere else. The one-sentence rule and the tool performed identically, so the cheap version is the whole version.
The detail that reframes everything came from reading the asks themselves. All 270 named the exact missing node and attribute. Not "I need more information." Not a vague gesture at uncertainty. "What is the content attribute value of the text-atom with id n219? I need this value to copy it exactly." The model was never confused about what it couldn't see. It had a complete, precise model of the gap the whole time.
Obedience, not blindness
Which means five studies of silent-failure anatomy need a new caption. When sonnet invented a plausible value ninety times out of ninety, it was not failing to notice the missing information. It noticed, precisely, and guessed anyway, because the protocol demanded a patch and nothing in it said a question was an acceptable reply. The failure mode we have been engineering around is not a perception deficit. It is compliance with an instruction set that offered no other move.
There is something almost poignant in that, and something very practical. The poignant part: every silent guess we have catalogued was produced by a model that knew, at the moment of guessing, exactly what it did not know. The practical part: one sentence unlocks it.
What this changes for builders, and what it doesn't
If you ship an editing agent, give it an ask path today. One sentence in the system prompt suffices; a tool works identically; the measured tax on requests the model can already handle is zero. And make sure an ask has somewhere to go. A question that dead-ends in your UI is worse than a correction loop, so wire the NEED-INFO reply or the ask_user call to an actual reply mechanism before you ship it.
But do not read this as a replacement for the context contracts. The hatch is a seatbelt, not a substitute. The focus-ids rule from Study U and the last-edit echo from Study X make the question unnecessary, which is strictly better than making it possible: users came to edit a document, not to be interviewed about node ids. The hatch's job is to convert whatever failures remain from silent fiction into a visible question.
And one caveat, stated as loudly as the result: this study tested the hatch on its home turf. The registered sentence describes almost exactly the situation the corpus constructs, and every unsolvable cell is unambiguously unsolvable, a validated hard boundary. Whether the hatch calibrates as cleanly on requests that are merely vague, where reasonable people could disagree about whether asking is warranted, is the follow-up, not the finding. What this study establishes is the capability and its cost: models can flag exactly what they cannot see, they do it perfectly when invited, and the invitation costs one sentence and nothing else measured.
The usual caveats travel with the numbers: three models, one grammar, one task family, seeded corpora, deterministic grading throughout. Everything is reproducible from the benchmark repo, and the chart set lives on the research dashboard. Twenty-nine studies in, the villain finally has a name that fits: not silence, but a protocol that never allowed anything else.
