Rich Sutton — the father of reinforcement learning and the author of the Bitter Lesson — has a new warning: the LLM industry is stuck in a local optimum, and more GPUs will not get it out. In an interview with Sequoia Capital, he argues that pretraining on human-recorded knowledge has a ceiling, that synthetic data is a "big mistake," and that the missing capability is continual learning — the ability to update a model's weights from its own experience after deployment. His new lab, Oak Lab, is betting this capability rewrites the entire "train-freeze-deploy" foundation-model paradigm, not just one generation of models.
The scaling trap: why the industry stays in a local optimum
Sutton does not deny the success of LLMs — he calls them a stunning scientific breakthrough. Language was one of the last strongholds of symbolic AI; neural networks trained at scale simply learned it. That is the Bitter Lesson working as intended: general methods that scale with compute beat hand-crafted human rules over the long run.
But the same logic now cuts the other way. In Sutton's view, the industry has fallen into a local optimum: stacking more GPUs and scaling further still improves benchmark numbers, so the labs at the top have less and less incentive to switch to a route that might perform worse in the short term. The better a company is at scaling, the harder it is to change course. Each additional scaling gain makes the exit more expensive.
The data ceiling: the internet is not the world
The deeper constraint is not compute but data. LLMs freed themselves from hand-written rules only to be bounded by a different kind of human knowledge: the internet.
Papers, code, books, pages, video, chat logs — everything stored online is what humans have already recorded. Sutton's judgment is blunt: the real world contains far more information than everything humans have ever put on the internet. Training on an ever-larger slice of recorded knowledge will eventually hit a wall, because reality always outruns the archive. Behind this sits his "Big World Hypothesis": the world an agent faces is always more complex than the agent itself, so no pre-collected dataset can ever cover it.
Synthetic data is a "big mistake"
The popular answer to the data ceiling is synthetic data — generate infinite training samples with enough compute. Sutton calls that a "big mistake," and says it may become the next bitter lesson.
His reasoning: synthetic data still routes through a human bottleneck. Who decides what is worth generating? Who judges which questions matter? Who defines the reward? At every step, a human decides — so the expansion speed of synthetic-data pipelines is still limited by what humans already know. It looks like an escape from the data ceiling, but it quietly re-imports human knowledge through another door.
This is one reason self-training loops matter: models that generate their own tasks and judge their own trajectories, like Ornith 1.5, are beginning to automate the question-creation bottleneck itself.
The missing loop: learning after deployment
Today's model lifecycle is artificially split in two: during training, the model learns intensely; once deployed, its weights are essentially frozen. Context can inject new information temporarily, and memory can store preferences, but Sutton's definition of learning is stricter: real learning means the system itself changes after an experience.
His examples are intuitive. A doctor who has done a thousand surgeries reasons differently than a fresh graduate. A driver with ten years on the road anticipates a lane-cutter without conscious thought. Experience has physically changed them. An LLM, by contrast, is a brain frozen at graduation — it works for years by rereading its own graduation notes.
So Sutton asks the central question: why can forming new concepts and modifying internal structure only happen during training? If changing weights is what produced intelligence, why stop doing it once the model goes live?
Catastrophic forgetting: why weights stay frozen
If continual learning is so clearly needed, why doesn't OpenAI or Anthropic just update weights daily? Because models can easily be "learned to death" — the classic failure is catastrophic forgetting.
Neural networks do not store knowledge in independent rows. A single weight change ripples across many abilities at once. Teach the model one new fact — say, "internal project X is now called Apollo" — and you may corrupt skills it already mastered. That is precisely why today's systems prefer stuffing new information into Context, RAG, or memory instead of touching the underlying weights.
Batch retraining, the way Cursor-style products do it, works for public models: collect millions of user traces, retrain once. But it breaks down for personal agents. If I want my own assistant to pick up a new habit, why wait for a hundred thousand other people's data? And what I want it to learn may be irrelevant to everyone else. This is the durable agent state problem in its hardest form — turning individual experience into lasting capability instead of a context window.
Step-size optimization: one learning rate per weight
Oak Lab's bet is on a low-level algorithmic fix. The direction, called step-size optimization, starts from a simple observation: not all knowledge deserves the same plasticity.
Knowledge that has been verified millions of times should be rock-stable; newly learned facts should be easy to modify. The practical mechanism: give each weight its own learning rate, so the model can absorb a single new experience without letting that update rewrite everything else. It resembles a human brain — an adult does not overturn decades of the concept of "cat" because of one green cat sighting.
What changes if continual learning works
If Oak Lab — or anyone — cracks online continual learning at scale, the rewrite is not just a better model. It is the end of the "train-freeze-deploy" paradigm:
- Foundation labs: the moat shifts. Data-plus-scale supremacy matters less when models keep learning from their own experience after shipping.
- Agent companies: permanent memory and experience accumulation become the product, not a feature bolted onto a context window.
- Inference infrastructure: serving becomes a loop — weight updates during operation — instead of static weights behind an API.
- Simulation vs. reality: if the world is always bigger than the training set, simulators stay useful but insufficient — wind, friction, motor error, and obstacles that never appear in any simulator are exactly what a drone meets on its first flight. World models help, but cannot replace lived experience.
What teams should do now
- Stop treating context, RAG, and memory as the final answer — they are scaffolding for a model that cannot learn. Design systems so that stable knowledge and rapidly changing knowledge are structurally separable.
- Measure forgetting first. Before adding any continual-update mechanism, build a regression suite of core capabilities and re-run it after every update.
- Watch per-weight plasticity research (step-size optimization, regularized continual learning). If it works, the winning architecture is not a bigger pretraining run but a model with a learning loop.
- For agent builders: start capturing structured experience today — what was tried, what failed, what changed the outcome — because that log is the training data of the next paradigm.
FAQ
Q: Why does Rich Sutton say LLMs are stuck in a local optimum?
A: Because more compute still improves current benchmarks, top labs have little incentive to switch to a route that may look worse in the short term — so every scaling gain makes changing direction more expensive.
Q: What is Oak Lab trying to build?
A: Agents that keep learning after deployment — updating their own understanding from experience, extracting new concepts, and adapting to a changing environment without forgetting what they already know.
Q: Why is synthetic data a "big mistake"?
A: Because humans still decide what is worth generating and what counts as correct. The pipeline is bounded by human knowledge again, just through another door.
