Ask HN: Are past LLM models getting dumber?

hmate9
2 days ago
4points
I’m curious whether others have observed this or if it’s just perception or confirmation bias on my part. I’ve seen discussion on X suggesting that older models (e.g., Claude 4.5) appear to degrade over time — possibly due to increased quantization, throttling, or other inference-cost optimizations after newer models are released. Is there any concrete evidence of this happening, or technical analysis that supports or disproves it? Or are we mostly seeing subjective evaluation without controlled benchmarks?
5 comments

Comments

muzani4 hours ago
I've been mostly using GPT-5 (the first) recently because it's better than 5.1 and 5.2. There's a clear difference. They're removing it next week; I'm canceling my subscription to ChatGPT.

This happens all across the board. Sometimes to Anthropic. Almost every time with Gemini. They optimize for some solutions and cause problems to others. Some people want a syncopathic AI like GPT-4o.

Personally, I loved the early patched GPT-5's ability to jailbreak itself. It was clearly a bug; great at finding solutions and skirting the absolute edges of the guardrails. They got rid of it because someone used this to override safety guardrails and commit suicide.

ChatGPT has nearly a billion active users. As long as they keep trying to please all of them, win the next 1 billion, and avoid being banned by governments, they're going to keep doing this.

The fix is to have your own model that works for a particular problem and self-host it or something. I still have a pet Mistral NeMo.

segmondy2 days ago
No, you're just getting used to them.
chistev2 days ago
Interesting.
rafiki62 days ago
No technical analysis, but all models experience drift eventually.