Lightning Jar - Web Studio Lightning Jar Wordmark

Study AL · Sessions & memory

The gate fails on a moved baseline; and unproven does not ship

The prompt-side fence

Study Overview

Measuring the Filed Fence

Study AK left one pathway open: the app-side eviction cannot restore a note the model pruned before sending, and the mid tiers pruned goals client-side in 4 to 6 cells of ten. AK's interpretation table filed the obvious candidate; one prompt sentence forbidding self-eviction ("never drop or trim an existing note to make room; the app decides evictions"); for measurement before shipping, and Study AL is that measurement, with the prompt rule as the only variable against the AK-eviction arm as its contemporaneous control.

Right Where the Gates Cannot See

The fence looked right everywhere the gates could not see: pooled client prunes fell 5 to 1, goal survival rose to 10 of 10 on sonnet and 9 of 10 on gemini, all sixty under-cap updates stayed untouched, and the sentence cost nothing; the models were already sending complete lists; the fence changed which notes survived, not how many tokens moved.

A Moved Baseline

But the pre-registered significance gate failed at p = .219, for a reason nobody registered: the control arm did not replicate its own three-day-old baseline. Sonnet's prune rate halved (4 of 10 to 2), gemini's too (6 of 10 to 3); same prompts, same corpus, same handler, temperature 0, the largest drift the series has caught between two measurements of the same construction. Against a five-prune pooled baseline, ten cells per tier cannot clear the bar.

A Double-Edged Verdict

So the published verdict is deliberately double-edged: the registered reading ("the prune pathway is not instruction-closable") overstates what this run measured, and the honest reading; unproven, not disproven; still ships nothing, because unproven does not ship.

What the Study Bought

What the study bought instead is sharper than a pass: the client-prune baseline itself is unstable week to week, which is the strongest argument yet for the filed alternative; an update shape that cannot express omission at all, so the guarantee stops depending on tier behavior holding still. Two smaller facts round it out: opus pruned for the first time in the series (a fact, not a goal; its goal safety stayed perfect), and consolidation-on-notice recurred, two sightings in two studies.