Findings No. 79
Part A (what detection knew)
Tempo confidence flags its own mistakes at about 0.84; key and meter confidence do not separate right from wrong at all
Not every confidence score a detector reports means the same thing. Stem Shovel's tempo, key, and meter detectors each emit a confidence value, but only one of the three actually told truth from error apart.
Across the wrong tempo detections we reviewed, a confidence threshold near 0.84 caught all but two of twelve errors, and both exceptions erred in the safe direction rather than failing wildly. Key confidence showed no such pattern: one waltz was called A minor at a confidence of 1.000 and was still wrong. Meter confidence separated nothing, right or wrong, across the sample. The candidate tempo ladder compounded the problem for a different reason: the correct tempo sat on the ladder for only three of the seven wrong detections. For the rest, including a hymn, a waltz, and a piece in 4:3, the right tempo was never on the ladder, so re-picking a different rung could not have rescued them; only a confidence-based refusal could, and testing that properly is its own registered study.
The fix that shipped treats tempo confidence as a real signal worth acting on around that 0.84 line, while key confidence and meter confidence are left out of any gating decision, because this sample gave them no separating power to act on.
The general rule: a confidence score is only as trustworthy as its measured separation between right and wrong, and that has to be checked signal by signal, not assumed to carry over from one detector to the next inside the same pipeline.
Receipt: Stem Shovel tempo/key/meter detection evaluation (12 tempo errors, including hymn, waltz, and 4:3 pieces; waltz detected as A minor at confidence 1.000).