The Truth Was Missing Notes

The last chapter was about chords. This one is about a single line. Stem Shovel keeps a song as its stems, one file per instrument, and the app wants each stem to carry its own part written out, starting with the bass: a plain-text line in barmark, the chart notation the app already uses, laid on the beats the app has already tracked. A bassist opening the song would see the notes, bar by bar. The question for stem-bench was the obvious one. Is a transcribed bass line right often enough to show a bassist?
Five studies later the answer is still no, but for the first time it is a close no, and the way it got close runs through a mistake in the bench's own answer key.
Six Songs With an Answer Key
Transcription studies usually have to argue about truth. This one did not, because the songs are the author's own. Six were made in Logic with a separate bass stem and the session's MIDI for the bass part: five by MMKK and one by Ghost Ghost, 599 bars and 2,299 notes in all. A committed intake script finds where each stem starts in MIDI time by matching onsets and pitch classes, confirms the offset holds through the song, and places the notes on the app's own timeline. The stems are Logic's rendering of that MIDI through a sampled bass, so the truth is exact, and this is the clean case: a bass played by hand into a microphone would do worse.
The grade is a bassist's grade. Each transcribed line goes onto the app's beats at a sixteenth or a triplet, and a bar counts as right only when every pitch, every octave and every slot matches, with no extra note. Before any run the bench registered what showing the line would take: at least 70 percent of bars entirely right and at most 10 percent wrong, on every song.
Right Notes, Wrong Bars
Study 48 ran three pipelines: the app's transcriber, Basic Pitch, as the app runs it; the same with a bass filter; and pYIN, a monophonic pitch tracker. Pitch turned out to be the smaller problem. Basic Pitch writes 13 to 38 percent of its matched bass notes an octave up, every error upward, and a simple bass constraint (the range E1 to C4, one note at a time keeping the lowest, nothing under 60 milliseconds) removes most of that for free. pYIN almost never misses the octave but finds fewer notes and grades below Basic Pitch on four songs.
The bars told the real story. Between 25 and 40 percent of bars were entirely right on five songs, and 72 percent on Letter Lands, a steady eighth-note part that still had 15 percent of its bars wrong. Quantizing to the grid was not where the line was lost: on four songs 82 to 95 percent of the notes landed in the right slot. The loss was onsets. The transcriber missed between one note in twenty and one in four, and added extra onsets numbering 8 to 32 percent of the part's. The answer was no for every pipeline.
Four Ways Not to Fix It
The next three studies went after the onsets with everything short of a new model.
Study 49 classed every wrong onset by a mechanical rule. About half the extra onsets are re-triggers, a new note starting while the same pitch is still sounding; almost none are ghost notes. The missed onsets are the short and legato ones: on the three songs that have them, a note under 130 milliseconds is missed 49 to 65 percent of the time. The study then made notes under 32 of the transcriber's own settings from one set of cached model outputs. Every loosened threshold hurt every song. Turning off the two passes that make notes without a detected onset helped all six, raising mean bars right from 41 to 48 percent, short of the ten points registered for a recommendation. Two filters on the notes, a re-trigger merge and a loudness floor, each lifted one song and swallowed real notes on another, because this bassist really does repeat sixteenths, and by spacing alone a re-trigger and a real repeated note are the same thing.
Study 50 asked whether the beat grid was to blame. The session MIDI carries the composer's own tempo and time signatures, a grid with no tracker in it, so the bench graded the same notes on both. Bars right averaged 48.5 percent on the tracker's grid and 48.1 on the composer's. The grid was not the bottleneck.
Study 51 tried showing only the bars that can be trusted: a bar is shown when a second reading of the stem writes it identically. Agreement with pYIN lifted the shown bars from 47 percent right to 76, and on two songs it would do. But it showed a quarter of the bars and a sixth of the notes, and on one song the agreed bars were wrong more often than right, because both readers dropped the same short notes. Trust by agreement is bounded by how good the second reader is.
Settings, filters, the grid and trust rules had all been tried. The transcriber itself was what was left.
A Reader Built for One Note at a Time
Study 52 ran a different kind of model. Basic Pitch is built to hear chords. The hf_midi_transcription package carries per-instrument checkpoints of a network designed for piano transcription, including one for bass trained on jazz double bass, MIT licensed. It ran as shipped on the same six stems, on a CPU, at about a third of the audio's length.
Its first scores made no sense. Note for note it beat Basic Pitch, yet on Letter Lands, the easiest song, it got 43 percent of bars right against Basic Pitch's 76. Looking at the bars it was charged with showed something odd: the extra notes it supposedly invented were the same notes, in the same places, that pYIN had written. Two unrelated readers rarely invent the same note in the same place.
The Truth Was Missing Notes
The fault was the bench's. When a bass plays two notes of one pitch back to back, Logic writes the second note's start before the first note's end, at the same instant. The parser that read the MIDI paired each end with the latest start, so it lost the first note and left the second with no length, and the grader then dropped that one as too short to be meant. Peaceful Dreams was missing 47 of its 454 notes, Interbellum 71 of 491, Downline 77 of 438: between 1 and 18 percent of each song's notes. Every bass study had been graded against that truth, and every transcriber that heard one of those notes had been marked down for adding it.
The bench's rule for a defective fixture is stop, fix, disclose, re-register and re-run. The parser was fixed and pinned by a test, every song's alignment was confirmed unchanged, Studies 48 to 51 were re-run, and their published entries were corrected where they stand, with the first numbers kept in each correction. Study 52 was registered again before its scored run, with the numbers from its defective first pass disclosed.
The re-runs changed numbers more than conclusions. Every decision held: still no line, no setting, no filter, no trust rule. But two sentences had been wrong in an instructive way. Study 49 had reported a class of extra onsets where nothing was sounding, 23 percent of them, and called the bench's earlier guess about re-triggers wrong. On the corrected truth that class is 3 percent and re-triggers are 49. The transcriber had been hearing real notes all along, and the grader had been scoring the answer key's gaps as the transcriber's inventions.
Sixty-Four Percent
On the corrected truth the bass model gets 53, 39, 51, 82, 74 and 87 percent of bars entirely right, a mean of 64 against Basic Pitch's 48 at its best settings. It makes no octave errors at all, none in 1,962 matched notes, with no filter, and adds about a third as many false onsets. Three songs, Eat All the Clocks, Downline and Mop Heart, clear the bar for showing the line. Nothing had done that on any song before.
By the rules written before the run it is still not recommended as Basic Pitch's replacement, because one song loses 25 points where two were allowed: Letter Lands, 51 percent against 76. The reason is again a question about the truth. That song's MIDI holds twenty notes four milliseconds long on the riff's fourth beat, the grader treats a note that short as not meant, and the model writes a note in nineteen of those twenty places. Counted as notes, the song would score 79 percent. The composer's ruling, given after the run: four-millisecond notes are debris from editing in the DAW and should be discarded. The truth is right, the sampler that rendered the stem played the debris, the model heard it, and 51 percent stands. A bass played by hand would not contain it, but the registered rule is not moved after the fact.
Seventy-One Percent
A second model ran as a registered amendment: MuScriptor, from Kyutai and Mirelo, a transformer that writes notes as a stream of tokens, told here that the instrument is an electric bass. It gets 55, 48, 85, 70, 75 and 89 percent of bars right, a mean of 70.5. It makes one octave error in 1,973 matched notes and a quarter of Basic Pitch's false onsets, and on Letter Lands it does not write the sampler's debris. Four songs clear the bar.
As a replacement for Basic Pitch it gains 22.1 points on the mean where ten were required, and no song loses. That is the first registered yes in five bass studies.
The bench's own bass constraint worked against it once. Peaceful Dreams goes below a four-string's lowest note, to D1 and C1, 29 notes in all. MuScriptor heard all 29, and the constraint, written in Study 48 to catch Basic Pitch's octave errors, removed them. Without it that song is 62 percent right and 18 percent wrong rather than 55 and 25. Neither new model needs the constraint, since they write one note at a time and make essentially no octave errors. The registered number is the lower one, and this is why.
Two costs come with it. MuScriptor's weights are licensed for non-commercial use only, so the app cannot ship them, and on this machine's CPU it took about four times the length of the audio. Mirelo sells a successor, Audio-to-MIDI Pro, as a paid service; the bench has not tested it.
Two Songs That Hold Out
Neither model makes a line that can be shown on every song. Interbellum is built from runs of short notes played without a gap, which every reader in these studies joins or drops: MuScriptor gets 48 percent of its bars right and 42 percent wrong. On Peaceful Dreams a fifth of the part's onsets go unfound. And agreement does not rescue them. The two new readers write the same bar four times in five, and where they agree they are right three times in four, which is no better than MuScriptor alone: on these two songs they are wrong together.
The Ledger
All five studies cost nothing, since every reader runs on the bench's own machine and every grade is mechanical. The predictions written before each run did badly early and well late: one of six held in Study 48, three of eight in Study 49, three of six in Study 50, all five in Study 51, five of six for the bass model and one of three for MuScriptor, whose miss in the useful direction was predicting it would score below the bass model. The corpus is six songs by one writer on a sampled bass. The truth was corrected once, for a parser defect that understated every transcriber for five studies. Published sentences that had blamed the beat tracker for bars no measurement tied to it were corrected in place, and so was a guess about ghost notes that Study 49 measured and found wrong.
The registrations, the graders and their tests, the corrections and the full report are in the stem-bench repository; the entries are Studies 48, 49, 50, 51 and 52 on the research dashboard. Four studies of settings, filters, grids and trust rules could not move the line, and a model built to follow one note at a time moved it sixteen to twenty-two points on the first try. The finding that made the result believable came from the model too. When a new reader is marked down for notes that an unrelated reader also hears, check the answer key before blaming the reader.