The warning heuristic, validated: the loop closes twice, at zero model spend

Validation

Study Overview

The Loop, Closing Twice

Study 5 caught the bench's own PDF renderer dropping accidentals; the correction brief suggested a production guard; Stem Shovel shipped it as chartReadWarnings (flag a read whose key signature calls for two or more accidentals while none of eight or more chord symbols carries one). Study 6 is the bench validating the production feature that exists because of the bench's own correction, and it cost nothing to run: the corpus already held 148 reader texts of the same engraved pages, 52 from corrupted PDFs and 96 from clean ones, with expected labels mechanical from the committed specs. The program's first zero-model-spend study.

Three Instruments, One Table

The shipped rule works where it matters: 10 of 10 detectable corrupted reads fired. Its false-positive class is exactly the one its own source comment worries about, now measured at 4 of 42 clean production-reader cells: honest sharp-key charts whose chords legitimately carry no accidentals trip the all-or-nothing rule. The registered refinement (count only chords on the signature's own altered letters) removes that class and exposes one the registration did not predict: modulating charts, where pooling signatures across sections makes one key's honest naturals look suspicious against another key's letters. An exploratory key-in-force variant, disclosed as post-hoc, judges each chord against the signature active at its measure and resolves both classes at 1 of 42, and its single surviving flag is the best result in the study: it fired on a read whose modulation was placed three bars late, a genuine defect flagged for the wrong stated reason. The misses on undetectable reads are floor effects, stub reads that never reach eight chords, per the registered prediction.

The Recommendation, and the Scorecard

Briefed back to Stem Shovel: adopt key-in-force scoping, re-validated on the app's side or a registered extension since the bench's variant is exploratory, and soften the warning's stated cause, because the evidence says the flag earns trust as review-this-read generally, covering both file corruption and a doubtful read. Scorecard: predictions 1 and 3 right, 2 half-right (the predicted fix worked; the modulation class was not predicted). Zero model spend, both registered instruments and the label derivation committed and tested before evaluation.