The Drums Get a Vote

Stem Shovel is a Lightning Jar side project that grew out of my side gig. I'm an amateur songwriter and musician and whenever I can find time I get together with friends to write and record music. As we've grown older and moved to different parts of the country, making music with my band is more of a remote collaboration. Stem Shovel is my attempt to build the tool I always wanted to remove some of the friction from the process of writing songs, structuring them, and recording them remotely.

When you record a song in modern DAWs, the session lives as a collection of individual audio files called stems. For example the drums are one file, the bass is another, each guitar and vocal lives on its own. Sharing those with the rest of the band has traditionally meant a folder in a cloud drive, a bounced MP3, and a text thread of "the bridge at about 2:10 is too loud". Stem Shovel is an open-source web app designed to organize and streamline that process. Upload stems and the song page plays all of them in sync in the browser, with a fader, mute and solo for each, comments pinned to moments in the music, the chart and lyrics beside the waveforms, and an idea recorder for the demos that come before any of that. It runs as a service for our own bands and for anyone else who wants it.

This article zooms in on one small feature in the app and the challenges we had to sort out to get it right: time signature analysis and detection.

Here's how it starts: while the stems are still uploading, the browser listens to them and fills in the song's tempo, key and time signature. Having these data points is important so that while the song is playing back the counter display can accurately show a count of bars and beats which is the meter a musician thinks in, rather than seconds and milliseconds. We try to automatically detect these data points as a convenience to the user so nobody has to type "120 bpm, D major, 4/4" into a form.

Tempo and key detection worked pretty well for us in our early prototypes but time signature detection turned out to be a little tougher nut to crack.

A Song in Four that Read as Three

The first version of the detector did the obvious thing. Each stem's audio was decoded and reduced to an onset envelope, a curve of how much new sound arrives at each instant (spectral flux over a 2048-point FFT, 125 samples a second). The envelopes of every stem were summed into one, as if you had mixed the song down, and the summed curve was analysed: find the tempo, then ask whether the beats prefer to group in threes or fours.

The grouping question is a comparison of autocorrelations. A rhythm in four repeats itself every four beats and every eight; a waltz repeats every three and every six. So you measure how well the envelope matches itself at lags of three and six beats, and at four and eight, and see which pair wins:

const four = at(4) + at(8);
const three = at(3) + at(6);
return (three - four) / Math.max(three, four, 1e-9);

Positive leans toward 3/4, negative toward 4/4, and the size of the number is how sure the envelope is. On synthetic click tracks it was flawless. On the first batch of real songs it was right too, until one of ours, a four-on-the-floor rock song that has never been anything but 4/4, came back as a waltz.

The culprit was not the drums. The song has a guitar riff built on a three-note figure, and once every stem was summed into one envelope, the riff's three-feel outweighed the drum kit's four. It was not a close call, either: the summed curve leaned 3/4 with the same conviction it brought to an actual waltz.

Why the Sum Loses the Pulse

The mixdown is a perfectly good way to listen to a song and a bad way to count it, for a reason that is obvious once said: summing envelopes throws away who played what. A drummer's kick landing on every beat produces an envelope that repeats at every lag, three, four, six or eight, identically. It carries the strongest pulse in the song and, in an autocorrelation contest, it says nothing at all, because it agrees with every candidate equally. The hi-hat and the bass, which do outline the bar, are quieter. And a riff that repeats every three beats is a strong, distinct, periodic signal that the comparison rewards handsomely.

So the sum is dominated by the loud parts that abstain and the periodic parts that mislead, while the parts that actually know the answer are outvoted by volume. Musicians resolve this every night without thinking: when the guitar plays a three-against-four figure, everyone looks at the drummer. The code needed to look at the drummer too.

One Envelope Per Stem, Voting Weighted By Confidence

The fix, in commit 7b91d22, was small. The feature extraction already had every stem's envelope in hand before it summed them; the combine step now keeps the parts alongside the sum. The meter detector runs the lean calculation on each stem separately, and then combines the leans as a weighted vote:

for (const x of envelopes) {
      const lean = meterLean(x, fps, bpm);
      total += lean * Math.abs(lean);
      weight += Math.abs(lean);
}
const lean = weight > 0 ? total / weight : 0;
const value = lean > 0.03 ? "3/4" : "4/4";

Each stem's vote is its lean times its own absolute value, divided at the end by the sum of the absolute values. That is a weighted average where the weight is the stem's conviction. A flat kick with a lean near zero contributes almost nothing in either direction: it abstains. A hi-hat and a bass with a clear four-beat pattern each lean firmly negative and carry weight. The riff leans positive and carries weight too, but it is one strong voice against two, and the average lands in four.

Two choices in those lines are worth naming, because they are judgment calls dressed as arithmetic. Squaring the lean (the lean * Math.abs(lean)) means a confident stem counts more than proportionally, which is the point: we want the rhythm section to win, not merely to participate. And the threshold is not zero. A song needs to lean 3/4 by three percent before the detector calls it a waltz, because most songs are in four and a tie should go to the prior. The comment in the source says it plainly: "4/4 is the prior: 3/4 needs a clear lean."

The result is reported with a confidence, which is just the size of the agreed lean. Our rock song now reads 4/4. The one waltz in the catalogue still reads 3/4. And the summed envelope still exists, because the tempo detector works better on the mix than on any single part: a tempo is a property of the whole, a meter is an argument between the parts.

When the Vote is Close Get a Second Opinon

A vote can be close, and a close vote is information. The song page treats a meter confidence under 0.15 as a shrug and, when the account allows it, asks an AI model to listen to the rendered mix as well. The notice says exactly what is happening, "4/4 time is a close call; asking the AI to listen too", and the answer arrives as agreement ("The AI agrees: 4/4") or as an alternative you can accept with one click, never as a silent overwrite.

The model is also the only listener that can answer outside the two-way race. The autocorrelation test decides between three and four; it has no opinion about 6/8 or 7/8, and a song with no fixed meter would just come back as a low-confidence four. The model is asked for the time signature as free text, including "free", and validated against a pattern before it is shown. There is a small engineering wrinkle in this handoff that a game-loop essay would appreciate: right after an upload the mix is still rendering in the background, so the request can come back 409 for up to three minutes. The page waits and retries rather than giving up, because the moment a user most wants the answer is the moment they just uploaded the stems.

The Tests are the Songs

Because the detector is a pure function over envelopes, the test file is a set of tiny synthetic bands. A click track in four at 120 bpm reads 4/4; a click track in three at 96 bpm reads 3/4. And the bug that started all this is a test now, named after the song it came from:

test("a three-feel riff over four-beat drums stays 4/4 (fleeing the capitol planet)", () => {
      // Hat and bass in four, a guitar riff in three, and a kick on every beat that says nothing.
      const parts = [
              extractFeatures(clickTrack(120, 4)),
              extractFeatures(clickTrack(120, 4)),
              extractFeatures(clickTrack(120, 3)),
              extractFeatures(clickTrack(120, 1)),
      ];
      expect(analyse(combineFeatures(parts)!).meter.value).toBe("4/4");
});

Its mirror image is there too: two waltz stems and a flat kick are still a waltz, so the kick's abstention is tested in both directions. Like the theory-quiz tests in Fifths, the file is where the domain rule lives. "The drummer decides" was a fact we knew from rehearsal; now it is a specification, and the three-percent prior and the squared weighting can never be tuned away by accident.

What The Vote Does Not Know

Honesty about the limits. The vote only helps when a song arrives as stems; a single mixed-down file has one envelope and one voter, which is the old behavior. It only decides between three and four. It listens to the first ninety seconds, which is fine for a verse-chorus song and wrong for one with a long free-time intro. And the weights are tuned on our own catalogue, a couple of bands' worth of songs, not the whole of music. The AI second opinion covers some of that ground, at the cost of a network call and a rendered mix.

But the lesson generalizes well beyond bars and beats, and it is the same one our toys keep teaching us. The obvious representation was the mixdown, because the mix-down is what a listener hears. The right representation was the parts, because the parts are what the band is playing, and the question we were asking, "what are they counting?", is a question about the band. When the sum gives a confident wrong answer, look for the information the sum destroyed.

Sign up for a free account, upload a song and try it out for yourself at www.stemshovel.com