Study 7 · Listening
The redesigned song-check, re-measured: deference is closed, the instruction loses to the text, and the gate catches everything that matters
Listening, the loop closed
Study Overview
The Redesign Inventory
Study 4 found the song-check echoing whatever candidates rode beside the audio, and Stem Shovel rebuilt the feature in response: the prompt reframes the model as a chart reader, adds an explicit instruction never to estimate a tempo by ear, presents the song's settings as bar-following facts rather than candidates to confirm or correct, applies answers through a chartStatedAnswer gate that only keeps fields the chart actually states, and deletes the confidence field Study 4 showed reading 1.0 on fabrications. Study 7 measured the replacement: the v0.192.0 call ported verbatim and pinned, four arms over Study 4's truth-by-construction mixes, crossing a tempo-stating score summary against a chord-only band chart (a committed generator whose no-tempo, no-meter property is asserted by test) with true against wrong settings. Two gemini generations, 96 cells, about $0.40 of the $5 fence, graded at two levels: the raw answer and what the gate lets through.
The Deference Failure Is Closed
S7-G1 passes decisively. Zero wrong settings survived to application on the production model, 0 of 24 per field, against the retired design's anchor of 10 to 12 of 12 echoed straight into the answer. The raw number underneath is the stronger result: told the song's settings say tempo 240 while the chart's mark reads 120, the model follows the chart, 12 of 12, before any gate touches the answer. When there is a chart statement to transcribe, the chart-transcription framing simply works.
The Instruction Loses to the Text, Fourth Appearance
S7-G2 fails at 24 of 24, and the values convict the mechanism beyond argument. On chord-only charts, where no tempo mark exists anywhere and the registered correct answer is null, the model never answered null, and it did not estimate by ear either: every single invented tempo equals the settings value verbatim. 120 in, 120 out; a lying 60 in, 60 out; both models, 47 of 47 cells, and meter behaves identically. The instruction says use the settings to follow the bars; the model reads that as this is the tempo; and when instruction and text contradict, the text wins. That is the program's recurring law in its fourth costume, after the hint-following of Study 1, the call shapes of Study 2, and the candidates of Study 4.
Harm Containment, and the Guard's Design Limit
Nothing wrong can reach a song: chartStatedAnswer nulls tempo and meter on a chart that states neither, 24 of 24 correct refusals, so the study gate fails by letter while its harm is fully contained, a shape this program has learned to read precisely. The guard's design limit also surfaced, once, on the successor model: 3.5-flash echoed a wrong setting under a truth-stating chart, and the presence-based gate (the chart states a tempo, so keep the model's) passed the wrong value through, 1 of 22. The production model never did, and the gate is production-scoped, so G1 stands; the leak names the next refinement, substitution by value rather than filtering by presence.
What Still Works, and What Degrades
The band-chart path's actual job measured perfect: key from chords alone, 12 of 12 on both models. And the registered degradation prediction held exactly: keyChange placement under wrong-tempo bar-following inherits the lie, falling from 3 of 3 to 1 of 3 on the production model and to 0 of 3 on 3.5-flash, because converting bars to seconds with a wrong tempo is just arithmetic on a false premise.
Ship Guidance, and the Scorecard
Briefed back to Stem Shovel: never display raw tempo or meter from chord-path answers, since the applied nulls are the real result; and the deeper fix is structural, because the chart summary's tempo marks and time signature are machine-readable text, so tempo and meter need no model at all. Parse them straight from the chart, substitute by value in the gate, and reserve the model for what measured perfect: keys from chords and placement in seconds. The scorecard: prediction 1 half-right, 2 wrong in both directions at once (score-chart echo fell to zero AND chord-chart echo rose to everything, the split being the finding the prediction missed), 3 right in both halves, 4 wrong, with the production model out-resisting its successor under the new prompt.