You cannot turn provider prompt caching off, so audit it instead

If your benchmark or cost model assumes caching is off, two major providers disagree. OpenAI automatically caches prompt prefixes of 1,024 tokens or more, and Gemini 2.5 and later ships implicit caching on by default; neither offers a request-level off switch. Anthropic's caching is opt-in. Any cross-vendor dollar comparison silently becomes a comparison of default caching configurations.

Since it cannot be disabled, Barkup Bench audits it instead: every call records the provider-reported cached-input reads, and a standing invariant (cache reads never exceed input tokens) held on all 46,258 calls audited. The published analysis then states which cells caching touched and which direction the economics lean, rather than pretending the meter was off.

The general rule: provider-side caching is part of the measured system. Record what the provider reports per call, assert the invariants that would expose accounting errors, and publish the audit beside the costs.

Receipt: Barkup Bench REPORT.md, "Addendum (2026-07-07): prompt-caching audit"; scripts/audit-cache.ts; the Barkup Bench dashboard.