One Model, Two Harnesses, a 36-Point Gap
On September 3, ARC Prize published a strange set of results for ARC-AGI-3, one of the hardest reasoning benchmarks available. The same model — GPT-6 Astra, at the same reasoning effort — was run through two different harnesses. In the standard harness it scored 62.7%. In the Provider Adapter harness, it scored 98.6%.
The cost curve is even more counterintuitive. The higher-scoring run spent less: roughly $26,100 down to $17,300, a 34% reduction.
The brain did not change. Only the way it worked did — and the agent effectively became a different level of system.
Days later, YC hosted a Paper Club session titled "Why the Harness Matters More Than the Model," featuring researchers from Prime Intellect, Stanford, and YC itself. The core conclusion fits in one sentence: the decisive battleground in AI is shifting from "whose model is stronger" to "who can organize models better."
For two years, nearly all attention — and nearly all capital — has gone into the model itself. That consensus may be breaking. The gap that decides winners is increasingly forming outside the model.
What a Harness Is, and Why It Was Dismissed
"Harness" originally means the rigging that transmits a horse's power to a cart. In AI, it is the entire runtime framework wrapped around a model doing real work: how system prompts are written, how context is organized, which tools are callable, where memory lives, how sub-agents divide labor, how sessions persist.
A useful distinction: the model determines how smart the brain is; the harness determines what that brain has in its hands while working.
This kind of work was long considered second-class in machine learning. Tweaking prompts, gluing on tools, writing task loops — it sounded like engineering, not research. Yet that dismissed layer now moves scores by 35.9 points. That gap is its value space.
YC partner François Chaubard's own experience is a footnote worth reading. In March, while trying Karpathy's automated research project, he only wanted a UI to see what the agent was doing. Search, experiments, review, writing, and progress management got wired in one by one — and he realized he had accidentally built a full harness. The model never updated. The papers kept getting better.
The principle is blunt: the model sets your ceiling; the harness determines whether you can reach it.
The boundary between harness and model is also dissolving. DSPy turned prompt tuning from a craft into a search problem. The Darwin Gödel Machine (DGM) edits harness code directly, lifting SWE-bench from 20% to 50%. Continual Harness folds lessons from past runs back into the next version. Optimization is marching inward, from the harness toward the model's own weights.
A Framework: The Three Layers of Agent Capability
Assemble the evidence from this session and a structure emerges. The capability gap between agents lives on three layers, and the outer layers are the most neglected:
Layer 1: The model itself. It sets the ceiling. But between frontier models, this layer is converging, and it increasingly behaves as a given condition rather than a differentiator.
Layer 2: How a single agent works. How context is layered, how tools are organized, where memory sits. Prime Intellect's Prime Agent is the exemplar: instead of scripting every step, it practices "information tiering" — current information lives in context, programs and results persist in a continuously running REPL, long-term memory sits in external storage and is fetched on demand.
Layer 3: How a fleet of agents is organized. Deployment, state management, permissions. This layer only reveals itself when agents enter companies — and it is where answers are scarcest today.
One critical caveat: a harness amplifies capabilities a model already has; it does not create them. On ARC-AGI-3, Prime's RHAE harness scored 95.5% with Claude Opus 5 — and only 25.7% with Terra. Same harness, different model, a near 70-point swing. The model is still the foundation.
Testing the Framework in Three Scenarios
Long-horizon tasks. Prime's team ran an agent through seven consecutive days of Factorio, a brutally complex automation game. The system spawned 633 sub-agents, generated over 23 million output tokens, and completed 24 of 196 technologies. At one point a serious mistake collapsed progress from five technologies back to one. The system did not wipe state and restart — programs, resources, and records survived, and the model re-planned from the new situation. For long tasks, the model decides the next move; the harness makes decisions continuous.
Local personal AI. Stanford's OpenJarvis ran a clean experiment: swap Claude Opus 4.6 for Qwen3.5-9B inside a Hermes Agent setup, change nothing else, and accuracy on two benchmarks fell by 24.8 and 38.8 points. Then they optimized only the five layers around the model — model choice, serving, agent design, tools and memory, continuous learning — and recovered roughly 56% and 77% of the loss. Across eight benchmarks, the best local configuration averaged 80.3% versus Opus 4.6's 83.5%, with four benchmarks at parity or better — at about one-eight-hundredth of the marginal API cost and roughly one-quarter the latency. Small models do not need to fully catch up before local AI becomes viable. The harness closes part of the gap first.
Enterprises. YC once ran 50+ Hermes Agents on virtual machines, each individually configured; debugging meant logging into instances one by one. Their internal harness "QM" centralizes conversation, context, and long-term state, turning VMs into resources agents call on demand. But scale exposed three deeper problems. First, "main character syndrome": agents treat the slice they can see as the whole system and propose changes that affect everything — human review is still required. Second, agents give up too early; YC's Grind Tool enforces a minimum runtime or token budget before an agent may declare failure. Third, and hardest, permissions: an agent inside Slack sees enormous context but cannot reliably know what may be shared with whom. YC keeps most database access read-only, with human confirmation for writes — though as trust grows, human approval itself risks degrading into a rubber stamp. The next enterprise bottleneck is not capability. It is a harness that can run a fleet and actually govern it.
In other words, the model layer is commoditizing while the orchestration layer accrues value. The pattern is not new: in the cloud era, selling servers and building orchestration (Kubernetes) were different companies' jobs; in the mobile era, chips and operating systems split into two layers. AI is repeating the same process — the value of the orchestration layer just got quantified for the first time by an ARC-AGI-3 scorecard.
There is also a practical corollary for enterprises: when 50 agents run inside one company, operations and permissions are themselves product capabilities. Whoever nails the third layer wins the enterprise deal — a point worth writing down for any team selling agent products.
What To Do With This
Actions differ by role:
- Technical decision-makers: evaluate harness quality alongside model scores. The same model varies by 36 points across harnesses; comparing model benchmarks alone misses the largest variable. Before paying for a stronger model on long tasks, check whether context management and state persistence are the real bottleneck.
- Agent builders: loosen your grip. Stronger models deserve less scripted flow, not more. Prepare resources — tiered information, a persistent REPL, external memory — and let the model choose its path. Guard against premature failure with minimum budgets rather than "retry three times."
- Team leads: do not defer permissions until something breaks. Set read-only boundaries and write-confirmation flows before agents touch Slack or databases, and assume human review will erode — design mechanisms that do not require permanent vigilance.
- Cost-sensitive users: a local small model with a good harness may already beat a raw frontier-model subscription on value. You do not have to wait for the models to catch up.
For two years, the industry competed on whose foundation was deeper. The next two years will be decided on top of it. The model determines how fast you can go; the harness determines how far.
Sources: ARC Prize ARC-AGI-3 results; YC Paper Club, "Why the Harness Matters More Than the Model"; Prime Intellect (Prime Agent, RHAE); Stanford OpenJarvis; YC's internal QM harness — as compiled in 36Kr's coverage of the session.