Study AA · Standing context
We tested our own headline and lost: strictness is not a capability gradient
Conflict resolution; the confirmation study that refuted us
Study Overview
The Setup
Study Z’s spec-conflict split was an accident with one conflict shape, so Study AA measured it on purpose: three registered conflict kinds (rule vs instruction, an explicit user countermand of a rule, rule vs rule), four arms (base, a priority meta-rule, soft “generally prefer” phrasing, rules restated in the memo tail), 432 cells.
The Prediction, Inverted
The confirmation gate predicted opus most literal and measured it LEAST literal; zero strict readings in 24 base cells against sonnet’s 10 and gemini’s 7, significant in the opposite direction (p = 0.016); and the same conflicted template that split opus 6/6 under Z’s three-rule packs read both-obligations 12/12 under AA’s four-rule packs: which reading wins is composition-sensitive, not a model constant.
Three Interventions, One That Works
The interventions: the priority meta-rule (“the user’s request takes precedence”) moved nothing significantly on any model; softening “always” to “we generally prefer” collapsed literal readings (sonnet 10/24 → 2/24, gemini 7/24 → 0/24); the cheapest fix that works; and memo steering replicated Study Z to the digit (sonnet 2/12 → 11/12, p = 0.004) while revealing a footgun: with a rule restated in the memo, opus enforced the ™ mark against the user’s explicit “written plain, no trademark symbol” 12 of 12 times (without the memo, all 36 base cells honored the countermand). Rule-vs-rule conflicts resolved by specificity, not styleguide position.
Study AB Closes the Footgun
The footgun is CLOSED, measured (Study AB, 144 cells): a one-sentence precedence clause inside the memo block; the production system’s shipped mitigation, tested verbatim; restored every trampled countermand (opus 0/12 → 12/12, p = 0.0005; sonnet 3/12 → 11/12) at zero cost to the memo’s steering benefit (satisfy-both stayed 12/12 everywhere), with the injury first replicated to the digit in the control arm.
The placement lesson: the same sentence class buried in the styleguide moved nothing; precedence language works at the point of injury, adjacent to the notes it governs.
The Bottom Line
Across 900 conflicted cells in Z, AA, and AB: zero violations, zero contamination; models resolve conflicted specs; the conflict itself is the authoring hazard.