“Claude Code burns too many tokens” has been one of the most common complaints in developer communities for the past six months — with no one keeping honest books. On July 29, agent infrastructure company Composio ran the numbers properly: same model, same tasks, and token consumption differed by up to 30 times purely because of which agent framework (harness) the model ran in.
This is not a complaint about one product. It is a signal: as model capabilities converge and prices collapse toward commodity levels, the cost battle for agents is migrating to the invisible shell around the model.
Three Experiments That Put the Account on the Table
Experiment one: Composio’s head-to-head — similar success rates, a 6× bill. Composio placed the same model, Kimi K3, into three harnesses — Claude Code, Hermes, and Kimi Code — and ran 28 identical tasks:
-
Success rates were nearly flat: Kimi Code 22/28, Hermes 21/28, Claude Code 20/28.
-
Token consumption split apart: median roughly 61K for Kimi Code, 67K for Hermes, and 340K for Claude Code — 6× Kimi Code, with individual tasks reaching a 30× gap.
-
In money terms (K3 at $3 per million input tokens, with agent workflows running ~95% input): about $0.22 per task on Kimi Code, $0.28 on Hermes, and $2.00 on Claude Code.
-
Speed diverged too: Hermes was fastest (median 179 seconds), Kimi Code the most frugal (297 seconds), Claude Code slowest (348 seconds).
In other words: the model performed almost identically everywhere, and the harness alone spread the cost 9×. To save money, swap the shell before you swap the model.
Experiment two: Sebastian Raschka’s observation — the excess is all input tokens. The well-known ML researcher reported the same pattern with Qwen 3.6: at comparable success rates, Claude Code often consumed 2–3× the tokens of other harnesses. One case he analyzed last month: a Claude Code run spent about 578K input tokens against roughly 4,500 output tokens across 25 turns. The problem is not a chatty model — the harness repeatedly feeds full conversation history, tool calls, command output, and file contents back into the model each turn, so input context snowballs.
Experiment three: Writer’s controlled trial — swap only the orchestration layer, cut cost 41%. Enterprise agent platform Writer ran the cleanest test: 22 fixed enterprise tasks across 6 fixed base models (Claude Sonnet 4.6, Gemini 3.1, Gemini Flash 3.5, Qwen 3.6, GLM 5.1, Palmyra X6), varying only the orchestration layer. Average cost per task fell 41% ($0.21 → $0.12), median latency dropped 44%, token consumption fell 38%, while task quality held roughly level (0.78 → 0.81). Quality per dollar rose 82%, and tasks completed per million tokens jumped from 54.9 to 92.0.
Three independent experiments converge on one conclusion: harness efficiency is now as decisive for agent economics as model capability.
The Frame: Harness = Agent − Model
Boris Cherny, the creator of Claude Code, offers a minimal formula: Harness = Agent − Model. The shell outside the model — interface, workflow orchestration, permission governance, memory, observability — is the harness.
Drop the experiments into that formula and two layers emerge:
On the efficiency axis, the harness writes your token bill. Once tool calls, retries, and multi-turn loops start spinning, cost compounds rather than grows linearly — some are already calling for a “harness tax” line on benchmarks. When models become utilities, the harness is the air conditioner that decides your power bill.
On the time axis, the harness has a six-month shelf life. In a July 28 YC interview, Cherny went further: the stronger the model, the thinner the shell. His advice to AI product builders — delete the entire system prompt, then add lines back one at a time and observe what each actually does — is an ablation study in product clothing. For Opus 5, he claims, most of that scaffolding is already unnecessary. One internal Anthropic example: hand Opus 5 an OpenCV binding and it draws portraits unprompted — nobody trained it to draw. This is model elicitation: the capability already lives in the weights, and the harness’s job is to release it, not to think for the model.
Stack the two layers and you get a counterintuitive conclusion: the harness matters, but it deserves no moat. It can save you 9× today and be absorbed by native model capability or flattened by open alternatives tomorrow. Cherny’s sharpest line: coding is solved — the real moat is domain, the vertical knowledge, data, and organizational structure around it.
Three Stages of the Agent Race
-
First half: the model race. Parameters, benchmarks, price. Essentially settled — open models matched closed ones, prices fell to cents per million tokens, and the model itself struggles to be a barrier.
-
Middle: the harness race. Orchestration efficiency, token economics, tool ecosystems. The Composio and Writer experiments are its starting gun — but the window is short, because harnesses have low barriers and iterate fast. Cherny’s “six-month shelf life” is already ringing true.
-
Second half: the domain race. Who owns vertical data, and whose organization can rebuild its processes around agents. In agent-assisted teams, product, design, data, and finance all write some code; information and decision flows compress. The moat starts growing in the organization, not the codebase.
For founders the corollary is harsh but clarifying: if your product’s value proposition is mostly “we built a better harness,” your window is six months to a year. Redirect toward data flywheels and industry know-how — that is how you survive the cost war.
Three Things to Do Immediately
1. Run an ablation on your own workflow. Trust your ledger, not benchmarks. Delete the system prompt and tool list entirely, add lines back one by one, and record the effect on success rate, tokens, and latency. You will likely find 80% of your scaffolding has no positive yield on the newest models.
2. Choose by cost per task, not price per million tokens. The sticker price is not the bill — the harness’s context management decides what you actually pay. Watch input-token share and retry rate: roughly 95% of agent workflow tokens are input, and that is where money leaks.
3. Fit circuit breakers to long-running tasks. Multi-agent loops and unattended overnight jobs are where the harness tax bites hardest: context bloat plus retry loops make bills grow non-linearly. Token budget caps and loop detection are far cheaper than reading the invoice afterward.
Models are the utility grid, the harness is your air conditioner — and what actually holds value is what you installed in the room. The second half of the agent race is about who does more work with the same electricity.
