The most expensive ingredient in training a coding model is no longer the GPUs — it's the problems. A coding model needs more than questions and answers: it needs a runnable environment, a test that judges right from wrong, and safeguards against the model peeking at old solutions. Difficulty has to be tuned — too easy and it learns nothing, too hard and it can't earn a single correct trajectory, burning the whole run. Historically humans did that curation: slowly, expensively, and perpetually behind.
This month an open-weight lab called Ornith handed that job back to the model itself. The result is a two-layer story worth unpacking: a genuine technical shift in how training data gets produced, and a hard lesson about how much of any benchmark claim you should actually believe.
What actually happened
In August 2026, Ornith released three open weights — a 397B MoE, a 35B MoE, and a 9B dense model, all under an MIT license. The headline change is inside the training pipeline: the model now generates its own training tasks, builds its own solving scaffolds, and runs its own solution trajectories — then all three go into reinforcement learning together. Problem-writing has moved out of human hands.
The numbers moved fast. On Terminal-Bench 2.1, Ornith 1.0 scored 77.5 two months ago; the new 397B scores 86.1 — a gain of 8.6 points, roughly 11%, in two months. In the same table Ornith reports Claude Opus 4.8 at 85.0. A freely downloadable MIT-weight model sitting above a closed frontier flagship — at least on paper.
The versioning tells the real story of the method: v1.0 learned how to solve problems; v1.5 learned what to practice. The harder skill is the second one — deciding which self-generated tasks are worth the compute. That is exactly where the human bottleneck used to live.
Then the verification gap
Here is where the story splits. The 86.1 is a self-reported number: Ornith ran it in their own environment with their own configuration, and the site notes it is a five-run average. Terminal-Bench 2.1 also maintains an official leaderboard where every entry is re-run and verified by the team. As of August 19, 2026, that board had 17 entries — and Ornith 1.5 was not among them.
Compare the same model across the two tables and the gap is stark. Anthropic's official submission — Claude Code + Opus 4.8 — verified at 78.9%, ranking fifth; the top spot is Claude Code + Fable 5 at 83.8%. Ornith's own table lists that same Opus 4.8 at 85.0 — a 6.1-point difference for an identical model.
Agent benchmarks like Terminal-Bench measure the whole stack: model plus scaffold, timeout, context window, and resource allocation. Change the configuration and the meaning of the number changes. The official board's submission rules even forbid modifying timeouts or resource settings — precisely because otherwise teams could inflate scores by tuning configs instead of models. The same lesson about who gets to verify a model's output showed up this week in a very different arena: a single AI model writing nearly a million lines of Lean to check a math proof.
Why this is a signal, not a footnote
Two structural shifts sit underneath this single release.
First, the data wall is starting to be self-service. The scarcest input in AI training is shifting from curated human data toward compute plus the quality of the verifier. If models can generate their own curriculum at scale, open-weight labs can iterate on coding capability at a pace that no longer depends on hiring more annotators. That is an AlphaGo-style self-play principle landing on the most commercially contested benchmark in the market — and it compounds the compute concentration we flagged earlier this week, where the winners are the labs that can afford both the compute and the verification loop.
Second, benchmark credibility is becoming its own battleground. When self-reported scores can diverge from verified ones by six points on the same model, "who measured it, and with what config" is now part of the evaluation — not a footnote. The verified leaderboard is becoming the currency, which is good for buyers: it puts the burden of proof back on the vendor.
What to do with this
- Evaluating coding agents? Check the official verified leaderboard, not the vendor's table. Ask which scaffold and config produced any score you are shown, and whether timeout or resource settings were touched.
- Buying a model? Demand a re-run. A claim that only holds under the vendor's own harness is a weaker signal than one replicated in your setup. Ornith's 86.1 is an interesting data point; it is not yet an independent one.
- Building models? Self-generated curricula are now a proven data-engineering pattern for coding RL. The moat is shifting toward the verifier — the eval environment and the quality of the tests — rather than the raw problem bank.
- Reading any 2026 benchmark headline? Apply the two-question test: was it verified by a third party, and did the configuration change?
The takeaway is neither "open source won" nor "the numbers are fake." It is that AI has begun producing its own training material — and that the industry's measurement system is racing to catch up with what that means.
