loading
Loading.loading
Loading.Notes on building and running software.
87 notes in Agency
Apple tested frontier reasoning models on puzzles of rising complexity and watched accuracy fall off a cliff past a threshold, even when the models were handed the exact algorithm. The critics have a real point about the setup. The lesson for your hardest problems survives both.
Past a point, adding tools makes an agent worse. The failure is not calling the tool; it is planning across a big tool ecosystem when tools fail silently. A 2026 benchmark across 1,665 tools watched one model fall from 51.90% to 11.36% as conditions got harder.
A model that nails a three-step task can come apart on a thirty-step one, and pass@1 never shows you the slope. A 2026 reliability study across 10 models and 23,392 episodes found software-engineering graceful-degradation scores fell from 0.90 to 0.44 over longer horizons. Reliability is not capability.
We write when we've shipped or learned something worth sharing. No filler, no fixed schedule.