Skip to main content
Lightning Jar - Web Studio Lightning Jar Wordmark

Findings No. 79

Part A (what detection knew)

Tempo confidence flags its own mistakes at about 0.84; key and meter confidence do not separate right from wrong at all

Stem Shovel Browser Platform verified

Not every confidence score a detector reports means the same thing. Stem Shovel's tempo, key, and meter detectors each emit a confidence value, but only one of the three actually told truth from error apart.

Across the wrong tempo detections we reviewed, a confidence threshold near 0.84 caught all but two of twelve errors, and both exceptions erred in the safe direction rather than failing wildly. Key confidence showed no such pattern: one waltz was called A minor at a confidence of 1.000 and was still wrong. Meter confidence separated nothing, right or wrong, across the sample. The candidate tempo ladder compounded the problem for a different reason: the correct tempo sat on the ladder for only three of the seven wrong detections. For the rest, including a hymn, a waltz, and a piece in 4:3, the right tempo was never on the ladder, so re-picking a different rung could not have rescued them; only a confidence-based refusal could, and testing that properly is its own registered study.

The fix that shipped treats tempo confidence as a real signal worth acting on around that 0.84 line, while key confidence and meter confidence are left out of any gating decision, because this sample gave them no separating power to act on.

The general rule: a confidence score is only as trustworthy as its measured separation between right and wrong, and that has to be checked signal by signal, not assumed to carry over from one detector to the next inside the same pipeline.

Receipt: Stem Shovel tempo/key/meter detection evaluation (12 tempo errors, including hymn, waltz, and 4:3 pieces; waltz detected as A minor at confidence 1.000).

Ask Eljay

Ask about the studio's work, research, packages, or writing. A few starters:

Answers come from this site's own content and link their sources. For anything that matters, email hello@lightningjar.com.