In the first week after a big launch, a new model breezes through a messy spec and writes clean code—then two weeks later it trips over a basic edit. The obvious guess is that labs cranked up power at launch for viral posts, then quietly nerfed the model to save server costs. Yet daily API trackers find no secret weight tampering, while the labs' own reports reveal a very different cause.
What actually broke at Anthropic and OpenAI
If nobody touched the model itself, why do even company CEOs admit quality dips? In his breakdown, Nate Herk points back to the GPT-5 launch: the day after release, Sam Altman openly confirmed the model had underperformed due to a request-router bug. The model's weights—the core numbers holding its knowledge—never changed, yet the traffic-routing layer failed.
Anthropic's official postmortem from April 23, 2026 shows this mechanism in even finer detail. That spring, users complained that Claude Code had "gotten dumber," while internal checks missed it. The investigation confirmed that API models and weights were untouched, but three bugs overlapped in the harness for Claude Code, Agent SDK, and Cowork. From March 4 to April 7, engineers lowered default reasoning for Sonnet 4.6 and Opus 4.6 from high to medium to cut long UI pauses, then rolled it back. From March 26 to April 10, the reasoning cache reset on every conversation turn instead of clearing once after an hour of idle time, causing the assistant to lose the thread. And from April 16 to 20, a bad system-prompt edit for brevity hurt code quality. Everything was fixed by April 20 in release v2.1.116.
Independent trackers like MarginLab and AI Stupid Level run the same API tests every day and see no secret weight decay. Herk himself notes there's no hard proof against labs, though launch settings may still be tuned aggressively for buzz. It helps to separate that personal hunch from verified numbers and look at the dial you actually control.
The hidden effort dial and a 5.3-point gap
A common mistake when judging a new model is comparing launch-day benchmark charts with everyday chat. Reasoning models spend internal tokens thinking before answering, and that thinking budget is controlled by an effort parameter with levels low, medium, high, xhigh, and max.
On the Claude Opus 5.5 launch page from September 22, 2026, Anthropic lists CursorBench 4.0 results on real multi-file coding tasks. At max effort, the model scores 57.8%, while at the default medium setting it scores 52.5%. Herk's video rounds the second figure to 52%, yet even exact numbers show a 5.3-percentage-point drop on the exact same weights.
In the table footnote, Anthropic explicitly notes that its headline records were measured at max effort, and Terminal-Bench 4.0 at xhigh effort. Everyday sessions default to medium for speed and quota savings. On simple prompts you won't notice, but on complex refactoring the model cuts corners.
How to set effort manually and build your own "nerf test"
To avoid relying on client defaults, set reasoning depth manually. Claude Code offers both a UI toggle and the /effort command: just type /effort low, medium, high, max, or auto to return to default. Cranking it to max on every minor edit doesn't pay off—replies take longer and burn your quota faster. low fits tiny edits, medium covers routine work, high saves you on hard debugging, and max isn't available everywhere and belongs only on the trickiest problems.
Your second safeguard is a personal control suite—what Herk calls a "nerf test"—so you don't have to guess by gut feeling after an update:
- Pick five real work prompts you actually run every week instead of abstract puzzles.
- On release day, run all five with a fixed
effortlevel and save the baseline outputs. - Re-run the same inputs every two weeks and compare completeness, accuracy, and error counts: a sharp drop at the same
/effortlevel immediately flags a client bug or a reason to switch models.

