Findings No. 18
A provider option that works on one model is an API error on its sibling
haiku-4.5 rejected the thinking configuration its production-tier sibling accepts: 64 API errors out of 64 cells. Pin the full call shape per model and verify with one cheap call before the expensive batch.
The call shape that runs perfectly on one model can be an outright API error on its sibling from the same provider. Stem Bench sent its production call shape, adaptive thinking settings included, to claude-haiku-4.5, and got 64 API errors out of 64 cells: the smaller model rejects the thinking configuration its production-tier sibling accepts. Zero scored data existed until the call shape was pinned per model.
Because the bench was pre-registered, the change could not be quietly absorbed: the per-model call shapes were registered as an amendment before any affected scored run, and the error batch is part of the record.
The general rule: a provider option is a per-model contract, not a per-vendor one. Pin the full call shape (reasoning settings, token budgets, sampling) for each model you run, verify it with one cheap call before the expensive batch, and treat any model swap as a call-shape review, not a string substitution.
Receipt: Stem Bench Study 1, REPORT.md Amendment 1 (commit 8c4201b).