For two years, the AI frontier has been defined by one question: how much smarter can the model get? That question is still open, but a second one has moved to the top of the industry's agenda — how much cheaper and faster can the machinery beneath the model run?
Recursive self-improvement (RSI) — AI designing and tuning the systems that run it — has quietly become the next frontier after reasoning and agents. OpenAI says RSI techniques in GPT-5.6 cut end-to-end service costs by 20% and lifted token generation efficiency by more than 15%. Anthropic reports that over 80% of its codebase is now generated by Claude. Jeff Dean left Google to found Discovery Loop, a company betting on exactly this direction, reportedly valued near $10 billion.
The pattern is unmistakable: optimization itself is being automated. And the most consequential move is where that automation lands next — the infrastructure layer, where AI begins to optimize the computers that optimize the models.
The RSI Wave Is Moving Down the Stack
Most RSI effort today lives at the model level: models proposing experiments, evaluating results, improving themselves. But a model is only as fast and cheap as the stack underneath it — operators, compilers, memory schedulers, distributed runtimes. That stack is still tuned by hand, mostly by a small number of senior engineers.
That is the asymmetry a new generation of companies is attacking. Qingmang AI (清昴智能), a Tsinghua-founded startup and reportedly the first Chinese company to focus on self-evolving AI infrastructure, has made "AI optimizes AI" its founding question — with a specific mission: make domestic Chinese chips usable at scale through software that improves itself. It connects to a broader structural shift we mapped earlier: the next layer of AI progress may not be a bigger model but better organization of the systems around it.
The Core Mismatch: L4 Models on L1 Infrastructure
Models are racing toward L4-style autonomy — agents that run for hours, chain tools, and plan across long horizons. Meanwhile, the compute base still needs L1-style manual babysitting: operator adaptation, compilation optimization, memory scheduling, framework porting, inference tuning. Every new model release and every new chip requires redoing much of that work by hand.
The problem is acute in China, where software ecosystems around domestic chips are fragmented: swap the chip and you re-adapt; swap the model and you re-tune. A single complex operator can take an expert weeks to develop — and can be made obsolete within months by the next model or chip iteration.
The token economics make the mismatch expensive. Complex agent tasks can burn hundreds of times more tokens than a traditional chat session, and some production workloads are already approaching trillion-token daily call volumes. Hand-tuning cannot scale to that curve.
How Self-Evolving Infra Works: Optimize, Then Evolve
Qingmang's technical lineage runs through Tsinghua professor Zhu Wenwu's lab, which formalized the idea of self-driven machine learning in 2022 — extending AutoML and meta-learning so that machines decide not just hyperparameters but tasks, data, strategies, and evaluation criteria. Founder Guan Chaoyu, a Zhu student and Tsinghua special prize winner, turned that line of research into an infrastructure company.
The system is built on two loops:
- Self-optimization — for "how to optimize this time": detect bottlenecks from real runtime state, try alternatives, keep what the feedback says wins.
- Self-evolution — for "how to optimize faster next time": accumulate every optimization run as reusable experience, so a new model or chip starts from prior knowledge instead of zero.
The compounding loop — optimize, accumulate, re-optimize — is the entire product.
Production numbers from real domestic-chip deployments: developing and tuning a complex operator dropped from about two weeks to roughly one hour, a ~336x speedup; full-chain model porting on domestic chips compressed from about a month to one day to one week, ~30x faster; representative operators run up to 30x faster than baseline. The company reports adapting dozens of domestic chips, covering more than half of mainstream Chinese accelerator models.
From One-Off Tuning to a Trillion-Token Factory
What changes is the delivery model. Instead of selling one-time chip adaptation services, Qingmang sells either tokens (high-performance, low-cost output) or the production line (the self-evolving token-production platform, private or operated). Customers never see the chips; they see cost, latency, and stability at trillion-token scale.
Adaptation itself becomes a continuously running process: a new model arrives and the system learns it; a new chip arrives and the system tunes it; a new workload appears and the system generalizes. Optimization compounds instead of resetting. This is what makes a "trillion-token factory" on domestic silicon possible — turning idle or inefficient compute into token capacity, which matters even more as bots now outnumber humans online and agent workloads keep multiplying traffic.
Why This Changes the Industry's Center of Gravity
Three implications are worth tracking:
- Optimization becomes infrastructure. The scarce capability shifts from "who can hand-tune kernels" to "who owns a stack that improves itself." That is a moat that compounds, not a service that invoices.
- Software becomes the chip multiplier. Where access to frontier hardware is constrained, self-evolving software is the lever that closes the efficiency gap. The ceiling on domestic chips is no longer the silicon — it's whether the software around it can keep up with the model.
- The economics generalize. Any heterogeneous or constrained fleet — not just Chinese ones — faces the same manual-tuning tax. Expect RSI at the infra level to become a global pattern, not a China-only workaround.
What Engineering Teams Should Do Now
You don't need to build a self-evolving stack to benefit from the direction. Four practical moves:
- Instrument everything. You cannot self-optimize what you cannot observe. Runtime telemetry is the raw material of the loop.
- Treat tuning as data. Record every optimization decision — what was tried, what won, what the trade-offs were — instead of leaving it as tribal knowledge.
- Close one loop first. Pick a single operator family or chip, automate its feedback cycle, then expand. Small compounding beats big one-shots.
- Buy the loop, not the benchmark. When evaluating infra vendors, ask whether optimization is a one-time deliverable or a continuous, self-improving capability.
The model race gets the headlines; the infrastructure race decides the margins. When machines start optimizing the machines, the teams that treat tuning as a compounding asset will be the ones with the cheapest tokens at scale.
