Study Q · Views & grounding
"Change every X inside Y" breaks everything; including the oracle
Fan-out edits
Study Overview
The Fan-Out Problem
One instruction, 2–32 targets, on the same large trees. Even with retrieval solved (every target visible in the view), models finish only part of the set: 62–69% overall, ~45% at 7+ targets, failures 100% partial coverage.
The Search Recipe Spirals
The 0.4 search recipe fails its gate (median 6 calls instead of 1; a third of runs over 100k input tokens; −24 pp vs full tree on gemini).
The First Model Inversion
And the series' first model inversion: sonnet is significantly better with a view (p = 0.022), gemini significantly better with the whole tree (p = 0.008); no context strategy is model-independent here.
Decomposition Fixes It
Study R then tested the fixes: prompt-side interventions (a worked fan-out example, a coverage checklist) do NOT rescue it; but decomposition, measured directly, is perfect: the app enumerates targets and issues one single-target edit per node, 90/90 tasks on both models (674/674 subtasks, zero failures) at ~⅓ the input cost of a whole-tree prompt. The flat line at the top is the recipe. Solid: sonnet-4.5; dashed: gemini-3.5-flash.