The Witnesses Were Right
Last week, the band was right and the benchmark's ruler was wrong, twice. This week the benchmark closed a study with a verdict we did not love: after three studies of failed detectors, the one remaining unknown in a chartless chart draft, which beat is bar one, should be collected from the user as a single tap. The app we were measuring wrote back a brief that amounted to an objection. A musician does not find bar one by weighing beat energy. A musician uses where the song begins, where it ends, and what the chords do. Measure those witnesses, it asked, vote among them with weights fixed in advance, and keep the humility rule: never move off a correct answer without a clear margin.
So stem-bench, our open, pre-registered benchmark series for the AI features in Stem Shovel, registered seven witnesses. Each one votes for a beat class, a position for bar one on the already-measured beat grid, or abstains with a stated reason: where the file starts, where the last strong hit lands, the distance between the two, where the chord changes concentrate, where the cymbal crashes live, what the drum and bass stems say, and where the song's sections turn over. The weights, the vote's margins and the gates went into the registration commit before any run, and they were never tuned afterward. The recordings are Badverbs' own, scored against the band's charts.
Three Witnesses Against One Ruler
The first run failed every gate it had. That sentence has appeared in this series before, and once again the failure was the finding. Time zero, the chord changes and the section boundaries agreed with the registered truth exactly on three songs. On the fourth, Eat All the Clocks, the same three witnesses agreed with each other, one beat before the registered truth. Three instruments that share no computation do not conspire. When they agree against your ruler, you audit the ruler.
The audit took an hour and told a familiar story. Eat's truth anchor had been fitted back in Study 25 by a hint-scoring instrument, against the same mis-copied chart whose fifteen phantom bars drove last week's correction, and it was never re-derived after that chart was fixed. Re-measured now, the hint instrument turns out to be nearly flat across a two-beat range on this song: 65, 66 and 66 bars of 69 at the registered anchor, one beat early and two beats early. It never had the power to pin the phase, and it had picked its winner on a margin of one.
The strongest witness had been in our own committed results for two days. The chart-informed alignment grid from Study 27 sat a constant 0.474 seconds from the truth grid on eat, on every bar, in two published studies. We had politely filed that away as eat's quarter-bar edge, the one song our alignment recipe could not quite land. It was not an edge. It was the ruler's missing beat. Correction 4: eat's bar one is at time zero, not one beat after it, the recipe was right on eat all along to a few hundredths of a second, and every eat reading in five published studies carries a dated correction on the page where it ran.
There is a detail here the app had predicted in its brief. These recordings are DAW exports, and a DAW bounces from a bar line. On all four songs the first audible sample now sits on or within hundredths of a beat of bar one. The witness our failed detectors never consulted was the start of the file itself.
Four for Four, Then Parity
On the corrected ruler the vote passed its hard gate: the true beat class on all four songs, none undecided, by margins of five to one, six to one, three to one and five to one. Better, the harmonic-rhythm witness alone went four for four, which is exactly the case the app's brief had asked about, since a single passing witness is simpler to ship than a vote. The honest post-mortem went out with it: the cymbal-crash witness scored one of four, the drum-and-bass stem witness vetoed itself on every song, and two witnesses abstained everywhere.
One more instrument defect surfaced on the way, and it is the kind worth writing down. A beat class names a position modulo one bar, and two anchors a whole bar apart share a class. Our first re-draft placed the voted anchor a full bar from the song's first bar, and the drafts collapsed below even a deliberately wrong grid, because every bar was being graded against its neighbor's answer. The registered fix is one sentence: bar one is the voted class's last bar line at or before the first sound. With it, the chartless pipeline produced the numbers this arc had been chasing since Study 28: Letter Lands drafted 71 of 75 bars, which is 94.7 percent, exactly the score the same model earns when the chart grid is handed to it, and Peaceful Dreams drafted 152 of 153, above its chart-grid reference. Chart parity without the chart. Tempo measured, length inferred, bar one voted, and nothing supplied by a human.
The Review Ran Both Directions
The best part of the day came after the results were committed. The app's agent, the one whose brief started the study, read the published numbers and sent back a review that caught two things the bench had missed in its own instruments.
First, our onset-envelope extractor silently caps at 90 seconds, and two of the four songs are longer. The witnesses that read a song's ending had never seen one; their abstentions were unmeasured, not measured. Second, the time-zero witness's four committed distances, 0.079, 0.058, 0.068 and 0.070 seconds at four different tempos, are one number wearing four disguises: the onset envelope's known tendency to report attacks about 0.07 seconds early, measured here by the one witness that does not read the envelope at all.
Study 31d registered both fixes and ran for free. With the cap lifted, the ending witnesses are now honestly dead on this corpus, abstaining on all four full songs because every one ends in a ritardando or an uneven tail. And with the lead removed, time zero is four for four at distances of 0.02, 0.03, 0.008 and 0.000 beats. The fourth witness the app ships is the first sample of the file, corrected by one constant.
Benchmarks usually point one direction: the bench measures the app. This arc ran the audit both ways in one day. The witnesses the app proposed exposed the bench's fourth broken ruler, and the app's reading of the bench's committed results exposed the bench's capped envelope and quantified its lead. Two agents, each mechanically checking the other's instruments, each catching what the other's authors had normalized.
The Coda
The ledger from this arc is short. A truth value is only as good as the instrument that fitted it, and an instrument that is flat across the answers cannot pick one; eat's anchor was set by a one-bar margin from a flat instrument and survived three studies unchallenged. Agreement among independent witnesses outranks any single registered truth, and the right response to that agreement is an audit, not a re-weighting. And the subject of a benchmark, given the committed briefs, results and instruments, is not just the thing being measured. It is the best-positioned reviewer the bench will ever have.
The vote does not ship yet. The humility screen went vacuous once the ruler was corrected, because no song's starting guess was right anymore, so the registered condition stands: two more songs join the corpus before the vote is trusted off the bench. The bar-one tap survives only as the fallback behind an honest undecided. The registrations, the four corrections, the void runs and the full report live in the stem-bench repository; the entries are Studies 28 through 31 on the research dashboard, with the Study 31 family's four committed stages on one page. The app is Stem Shovel, the recordings are Badverbs' own, and the witnesses they carry were, on the question that mattered, exactly right.
Postscript, Later the Same Day
The coda's condition resolved faster than expected. The band had no fifth and sixth songs ready, so the app's agent wrote the substitute itself: a brief full of songs built to fool the witnesses. Renders where bar one is already right and no witness may move it, starts that are not a bar line, a one-beat pickup, a half bar of silence, a count-in click, harmonic rhythms designed to trick the chord witness, a backbeat-heavy mix, waltzes and 6/8, and the four real recordings re-cut with pads and trims. The bench adopted the brief whole and ran it as Study 32, twice, because the first run caught the bench's own synthesizer making a flux-invisible kick drum. With that fixed, the vote kept every correct bar one, never confidently decided a wrong one on any synthetic adversary, moved a wrong backbeat base to the truth five to nothing and kept a correct one five to nothing, and on the real re-cuts went fifteen right, four honestly undecided, and one wrong. The one wrong earned its keep: a cut that starts between beats silences the vote's two strong witnesses at once, and that condition is detectable, so the rule ships as keep the declared grid or ask. The app shipped the voted grid to its development branch the same day. The real fifth and sixth songs remain the confirmation when they arrive.
And one sentence above needs correcting, because the band corrected it. This postscript's author wrote that every song "ends in a ritardando or an uneven tail." That was the instrument's label, not the music: no song in this corpus slows down at the end (the one Badverbs song that genuinely does, Interbellum, is not in the corpus). Measured again with the band's correction in hand, the endings are fills and running eighths played right on the click, eat's final bars are metronome-exact eighth notes, peaceful lands its last hit three hundredths of a beat from the grid, and every song rings for a fraction of a second of lingering sustain afterward. The ending witness was mislabeling busy endings as slowdowns, which is a sharper reason to leave it out of the vote than the one we published, and a fitting last entry in a story about instruments whose labels deserve an audit.
