Findings No. 12
An LLM call can return an empty completion deterministically
One prompt returned 5,976 tokens in and zero out, twice in a row. An empty completion is a real output class distinct from errors; one retry is cheap insurance.
One prompt in Stem Bench's corpus returned 5,976 tokens of input and zero tokens of output, twice in a row. Not an error, not a timeout, not a refusal: a clean, reproducible, empty completion from a frontier model. Under the study's registered no-retry rule it scored zero on all 28 bars of that song, which is exactly what pre-registration is for, and exactly what production code cannot afford.
The production response shipped the same day: the chart-draft pipeline gained one retry on an empty completion (Stem Shovel v0.142.0), and the bench re-measured the shipped configuration under a registered amendment.
The lesson is that an empty completion is a real output class, distinct from errors, that deterministic inputs can deterministically produce. If your pipeline parses the completion, it needs a branch for the completion that does not exist, and a single retry is cheap insurance against the rare prompt that lands in this hole.
Receipt: Stem Bench Study 1, REPORT.md results footnote and "Production response and Amendment 2"; Stem Shovel v0.142.0.