How a Chart That Wouldn't Parse Became a Standard

stem-bench is our open, pre-registered benchmark series measuring the AI features in Stem Shovel, our music collaboration app. The last two articles in this series arrived at the same sentence from different directions: the binding constraint on machine musicianship was not hearing, and it was not the models. It was the quality of the chart.

This is what happened when we took that sentence seriously. The story runs through four studies and one band. It passes through a grammar argument conducted entirely in questions and answers. And it ends with the app deleting its own chart parser in favor of an open standard that did not exist the week before.

The Problem Was Always the Chart

Study 12 showed that fifty-year-old alignment math can place a chart's bars against a real recording to the hundredth of a second, wherever the chart actually describes the performance. It also showed the other half of that sentence: of the band's three public charts, one could not be transcribed at all and one was wrong in the middle.

Study 13 measured the chord pipeline falling from 96 percent on synthetic audio to 38 percent on real recordings.

Both studies pointed the same way. The remaining problems were data problems, and the data was the chart.

Two Constants That Almost Shipped

Study 14 ported the app's not-yet-released alignment code and caught a constant before it reached production. The code declared a chart and a recording "the same piece" when aligning against a scrambled control cost three times more than aligning against the chart. On synthetic audio the right charts scored 24 to 49, so a threshold of 3 looked generous. On the band's real recordings the right charts scored 2.0 and 2.5. The shipped constant would have rejected the band's own correct charts. We recommended 1.6, measured from real wrong pairs, and the release was held until it landed.

Study 15 registered the patch and watched the margin collapse. With a third real song in the wrong-pair pool, two wrong pairs landed at 1.57 and 1.58, two hundredths under the gate. The save came from a different signal entirely: the median of the per-bar local alignment cost. That number separated cleanly, 0.17 to 0.24 for right charts against 0.37 to 0.48 for wrong ones. We recommended a second clause, median local cost at or below 0.30, joined to the ratio with an AND. Two checks that fail in different directions restore the margin that either one alone was losing.

Study 15 also carried a quieter milestone. The band rewrote the Peaceful Dreams chart by hand, and for the first time a chart in the catalog was fully machine-true: 152 bars, ten sections, every section start landing exactly on the declared grid.

Writing a True Chart Forced a Grammar

Writing a machine-true chart surfaced every ambiguity in how the band had always written. What do three symbols in one bar mean? How do you say that everything drops out except the drums? Is G5(7) a legal chord name? How do you notate a final chord that rings past the last bar and dies somewhere unmeasured?

Each question got an answer. Each answer became a rule. The rules accumulated into something larger than the app:

• N.C. means no harmony, not no sound.
• Three dots alone in a cell mean an unmeasured ring-out, not a bar.
• A bare repeated section heading replays the section, so a song's structure reads like its set list.
• A parenthetical after a chord adds a tone, and a 5 before it keeps the third out, exactly as Logic Pro writes it.

At that point the notation stopped belonging to the app. We named it barmark, bar plus markdown, and gave it a home at barmark.dev, where the syntax is defined, with a user guide and an interactive playground for trying charts in the browser. The binding grammar lives in the barmark repository, and a reference parser is published to npm with provenance attestation.

A barmark file is a markdown file, so every toolchain on earth already renders it tolerably. ChordPro remains the standard for lyric-anchored songs. barmark is bar-anchored: the grid musicians already write between pipes, specified precisely enough for machines.

The App Retired Its Own Parser

Stem Shovel then did the honest thing, which is also the unusual thing. It retired its own parser.

The standard's parser went in behind a thin adapter. A migration converted the stored charts under a refuse-not-guess rule: the script converted three of five cleanly and refused two, which the band finished by hand, each confirmed against its recording's section markers. The standard's renderer took over the chart display. The in-house parser was deleted.

Study 16 graded all of it, registered before any run, at zero dollars. All four gates passed and all four registered predictions hit.

• Parity. The old parser and the new one read every committed chart bar for bar identically, 76, 152 and 86 bars, with zero warnings on either side. Nothing the app could read was lost.
• Migration. Every public migrated chart lands every band-declared section start at 0.0000 seconds on its own grid, including a 3/4 song at 224 beats per minute with a key change in the middle of the chart.
• Placement. Alignment did not regress through the entirely new parse path.
• The fix. Eat All the Clocks, the song whose chart Study 12 flagged as wrong, went from a median section error of 1.99 seconds to 0.09 seconds once the band corrected the chart.

The claim that the chart, not the aligner, was the bottleneck is now proven by the fix.

The Pair That Beat the Gate

One result in Study 16 deserves its own section. In an exploratory set of wrong pairings, the chart of one song aligned against the recording of another scored a cost ratio of 1.74. That is above the 1.6 gate. By the ratio alone, production code would call a wrong chart the same piece. The second clause caught it: a median local cost of 0.323, over the 0.30 line.

This is the failure mode Study 15 predicted, arriving one study later as a real catalog pair. As a band's charts get truer, a wrong chart looks more like a right one by total cost, because everything in the catalog shares a tonality and a feel. The local-cost clause asks a different question, how well each individual bar matches where the path put it, and that question still separates. The recommendation went back to the app the same day, upgraded from prophylactic to urgent: AND the second clause into the shipped check.

The Loop, Named

Here is the shape of the thing. The app shipped a feature. The bench measured it against the band's recordings and found the charts wanting. The band fixed its charts, and fixing them forced the notation to become precise. The precise notation became a standard with its own parser. The standard's parser went back into the app, and the band's recordings then validated the whole loop, mechanically, against the band's own declared truth. Every party in the loop made every other party better, and the only judge anywhere in it is committed code.

The ledger for the arc: Study 14 cost about forty cents, Studies 15 and 16 cost nothing, and the program's sixteen core studies together remain under twenty dollars.

The honest accounting, as always. Study 16 took one amendment: the first run crashed in an exploratory extra when production code refused a chart twice as long as a recording, which is the right answer delivered the wrong way, so the runner now records the refusal. One of the five migrated charts lives on no public page and is disclosed as out of scope. And the corrected Eat chart has no committed content truth yet, so its harmony was reported, not gated.

The recordings are by Badverbs, who share their stems publicly on Stem Shovel. The analyst model behind this series is Claude Fable 5; every grade in every study is computed by mechanical graders committed before any run, so no model ever judges an answer, including its own.

Where Everything Lives

The registration, the graders, the migrated-chart corpus and the full report are in the stem-bench repository; the Study 16 page is live on the research dashboard. The barmark syntax, user guide and interactive playground are at barmark.dev; the grammar and reference parser are on GitHub and npm. The series neighbors: The Last Job and Thirty-Eight Percent. The app is Stem Shovel.