The Pre-Training Wall and the Treadmill After It — CoRecursive #121

Host Adam Gordon Bell and guest Don McKay trace how frontier labs hit the "pre-training wall" — the point at which scaling laws stopped delivering free capability gains from more data and parameters — and what's replaced it. They explain why OpenAI's valuation still holds up against open-weights competitors and leaked checkpoints, and how reinforcement learning and model distillation have become the new axis of competitive advantage. A useful primer on AI moats, the Red Queen dynamics frontier labs now run on, and why "we have one internet" matters. Links to the readable episode transcript.