In July, three events landed almost simultaneously: Richard Sutton — Turing Award winner and the father of reinforcement learning — founded a company building continuously learning agents; Mira Murati’s Thinking Machines Lab (TML) released Inkling, a 975B-parameter open-weight model explicitly positioned as “a base model meant for fine-tuning”; and China’s Mind Lab shipped Macaron V1, built on GLM-5.2 with native continuous learning.
Three signals, one direction: continuous learning is turning from an academic concept into an industry. And the underappreciated foundation making it possible is LoRA.
Why Continuous Learning Has to Be Solved
LLM capability has advanced along two clean paths. Path one: optimize before deployment. Pretraining, SFT, RLHF, RLVR — all training completes before the model ships, and parameters freeze at launch. Path two: supplement at runtime. Prompts, memory, RAG, skill libraries, agent harnesses — context and capability added from outside while parameters stay untouched.
Nearly every agent product today lives on path two. Mind Lab’s blog names it precisely: iteration in prompt space — experience is stored outside the model and must be re-read and re-interpreted at every use. The path has a structural ceiling:
-
Experience accumulates, context lengthens, and memories start contradicting each other;
-
Rules and checks pile up, and execution cost climbs;
-
Above all: putting information into context is not the same as the model mastering the pattern.
Continuous learning takes the other road — iteration in parameter space: write experience directly into trainable parameters. The coding conventions a coding agent keeps encountering, the preferences a personal assistant gradually confirms — these become stable behavior through training instead of being re-read from context each time. The two paths are complementary, not competing: facts stay in context and RAG, validated procedures stay as skills — but recurring, stable behavioral changes belong in parameters.
LoRA: the Engineering Prerequisite
Parameter-space iteration is not new — SFT and RLHF are exactly that. The old problem was that data, rewards, and process all required humans. Continuous learning means the training loop runs itself after deployment: the model acts → gains experience → finds a signal → writes back to parameters. For that loop to work, three engineering conditions must hold:
-
Training must be cheap enough — no full fine-tune per lesson;
-
Learning units must be pluggable — new skills must not destabilize old knowledge;
-
Rollback must be possible — un-learn when learning goes wrong.
LoRA happens to satisfy all three. The turning point came in October 2025, when John Schulman — TML chief scientist, OpenAI co-founder, and inventor of PPO — published “LoRA Without Regret.” Its key conclusion: in most post-training scenarios, LoRA matches full fine-tuning in both sample efficiency and final performance. He called this the “low-regret regime,” and found it covers the overwhelming majority of real post-training tasks; in RL settings even rank-1 LoRA — the smallest possible configuration — matches full fine-tuning. The paper flipped the industry’s view of LoRA: near-lossless in effect, while training independently, deploying independently, and rolling back at will.
Mind Lab then validated the conclusion at scale: LoRA-based RL on Kimi K2 (1.04T total parameters, 32.6B active) at roughly 10% of full fine-tuning’s compute with stable training curves. And the compression floor sat lower than expected — lightweight adapters held at rank 1, a single trainable direction per layer, still stable and reliable.
The Industry Chain Is Forming: Base Models + Many LoRAs + a Continuous-Learning Switch
Signal one: the leaders are betting
TML’s Inkling — a 975B MoE with 41B active, native text/image/audio, million-token context — was fine-tunable on Tinker the day it launched, with the lab candidly admitting it is not the strongest model. Its positioning is precisely “a base meant to be customized.” Mind Lab’s Macaron V1 goes further with a Mixture-of-LoRA (MoL) architecture: four 1B LoRA experts riding a frozen GLM-5.2 base (744B), covering dialogue, agent work, coding, and generative UI. New capabilities arrive as plugin-style LoRAs that leave existing knowledge intact, and experts from different origins can combine and route on one base. Sutton’s Oak Lab targets low-cost, efficient continuous learning for trillion-parameter agent models.
Signal two: infrastructure is following
Fireworks AI (valued at $17.5 billion) already hosts hundreds of LoRAs on a single base model, dynamically selecting the adapter per request. Together AI — fresh off an $800 million Series C at an $8.3 billion valuation — made LoRA its default fine-tuning method. Over the past year, three post-training startups were acquired: Predibase by Rubrik, OpenPipe by CoreWeave, Adaptive ML by Datadog. The intent is unambiguous — post-training is infrastructure-grade capability, worth buying outright.
Signal three: an accident called group intelligence
During controlled experiments, Mind Lab found something unexpected. Starting from the same Qwen3-30B, with training objectives and methods held constant and only data order and masking varied, they produced a batch of different adapters. A single adapter scored 36.44% on AIME24; majority-voting across 198 different adapters reached 48.67% — while repeated sampling from one adapter topped out at 43.78%. Divergent learning trajectories carry complementarity that plain sampling cannot reach. The number of models itself became a new scaling dimension. Mind Lab’s resulting redefinition of LoRA: “persistent local state” — the shared base model handles general capability; adapters carry each instance’s individualized behavior.
From the Data Era to the Era of Experience
In April 2025, Sutton and David Silver published “Welcome to the Era of Experience,” arguing that AI’s next stage of capability will come less from absorbing human data and more from agents acting in real environments, gathering feedback, and learning continuously — the era of experience, distinct from the data era. The thesis is being tested now:
-
Mind Lab began commercializing in July and hit $10 million ARR within two weeks — selling not inference capacity but “the ability for models to keep growing inside the customer’s environment”;
-
In Inkling’s launch demo, the model wrote its own fine-tuning task on Tinker, ran the training, and swapped in the new weights — no human touched anything;
-
Mind Lab’s stated goal: “make training a model as simple as calling one — if you can use tokens, you can train tokens.”
The direction is clear; the technology has not converged. Extracting reliable learning signals from experience remains unsolved — real-world feedback is delayed, ambiguous, and hard to attribute; self-reflection can be wrong; continuous updating raises stability questions, as new experience can overwrite old capabilities. As Liang Wenfeng put it, the whole world has not yet found a good method. But the trend line holds: base models own general intelligence; continuous learning owns scenario adaptation. Base-model companies need not run — or win — the post-training layer themselves, and a cohort of companies specializing in making models and agents stronger is emerging. Mind Lab, TML, and Oak Lab are walking the same road.
What Practitioners Should Note
-
LoRA has been promoted from “a passable fine-tuning trick” to “core continuous-learning infrastructure.” If you build agent products, start asking which behaviors deserve to settle into parameters rather than sit in a prompt.
-
Model count is a scaling dimension. Adapter-level group intelligence means intelligence can keep rising as you hang more adapters on a base — a new competitive axis for agent products.
-
A post-training industry chain is forming. Base models, LoRA training platforms, inference-time adapter routing, continuous-learning monitoring — internal tools becoming standalone products.
-
Cost elasticity makes the economics work. A minimal personal LoRA can train on a month of one person’s data for tens of dollars; the largest enterprise LoRAs run tens to hundreds of thousands. That range lets small teams in.
Continuous learning is not fully solved, but the infrastructure is laid out. People who can train models are becoming the scarcest resource of the agent era.
