In May, Meta drew a red line for its own engineers: the Applied AI Engineering division could no longer freely use Claude Code and Codex. Note who is restricting whom. Meta is one of Claude Code’s largest customers on the planet, and its internal AI bill this year is racing toward the billions of dollars.
A tool the company depends on daily, purchased at enormous expense, now restricted internally. Not because the tools are bad — because they are too good. Good enough that Meta got scared. And behind this one policy decision hides an awkward question the entire AI industry is confronting in public: when AI starts helping us build AI, can anyone still tell whose intelligence a given capability actually is?
What Exactly Is Meta Afraid Of?
According to internal documents obtained by The Information, Meta’s core worry is not cost. It is distillation. The mechanism is easy to grasp: take a stronger model — say, Claude — have it answer questions endlessly, then use those answers to train a weaker model, say Meta’s in-development MetaCode. It resembles a top student redoing the exam from scratch so a struggling classmate can copy it — and close years of accumulated skill in months.
What Meta fears is not engineers writing code with AI. It is where the output goes. If Claude-generated code becomes MetaCode’s training data, then MetaCode learns not “what human engineers know” but “what Claude knows” — copying off the rival’s exam until it resembles the rival.
More insidious still is the evaluation loop. Every answer a model gives needs a judge to say whether it was good. If the questions and the grading both come from Codex, then MetaCode evolves toward whatever Codex considers correct — the opponent’s judgment standards, replicated bit by bit into your own model’s head.
Meta’s internal guide drew three red lines, each sealing the same leak. First: Claude or Codex output may not be used to write test questions for Meta’s own models — “we don’t want model-derived tasks,” in the guide’s words. Second: AI may not hunt for bugs in source code, nor propose what to test based on code analysis. Third: nothing AI-generated may be placed anywhere the evaluated model can access. What remains permitted: building workflows, organizing code, writing test scaffolding — the assistant’s job. But every line of output passes human eyes first.
An Industry-Scale Awkwardness
What Meta fears has a name in the industry: the distillation trap — the more you rely on the strongest model to build your foundation, the harder it becomes to prove where your intelligence came from. And the problem is not Meta’s alone. It has been surfacing everywhere.
In February, Anthropic publicly accused DeepSeek, Moonshot, and MiniMax of an “industrial-scale distillation attack” — mass fake accounts pumping carefully constructed prompts at Claude to systematically extract its capabilities. That was the visible tip. On June 10, in a letter to the US Senate Banking Committee, Anthropic disclosed that operators tied to Alibaba and the Qwen AI lab had run roughly 25,000 fabricated accounts through more than 28.8 million conversations with Claude — the largest distillation operation Anthropic has ever disclosed publicly.
Not even Musk stands outside this. On April 30, on the witness stand of a California federal court, the xAI CEO was asked whether distillation from OpenAI models had been used to train Grok. He first answered that it is common practice among AI companies — then, pressed, conceded “partially, yes.”
Here is the uncomfortable legal reality: in the United States, distillation is not illegal. AI-generated content carries no copyright. The only binding constraint is contract — OpenAI’s and Anthropic’s terms of service both forbid using model output to build competing products. But the “enforcement power” rests entirely with the rival who was wronged, with no neutral arbitrator anywhere in the system. Last year, Anthropic simply cut off OpenAI’s API access to Claude. When the rules are vague and enforcement belongs to your competitor, who would bet billions of investment on the rival’s goodwill?
Rented Intelligence Versus Trained Intelligence
Underneath sits a more fundamental question. AI companies have long demonstrated capability the same way: leaderboards. My model beat yours. But who wrote the questions on the leaderboard? Chatbot Arena and similar human-preference evaluations use blind votes from real users, which is relatively trustworthy. But a large share of technical benchmarks — coding suites, math reasoning sets, exam banks — how exactly were their questions generated? How many are organically human, and how many were distilled from another model’s output?
Once that provenance question cannot be answered, the whole capability-claim system cracks. Imagine that every AI model’s coding skill is evaluated against a question bank generated by Claude. What does a perfect score mean? That the model learned to program — or that it learned to imitate what Claude considers a correct answer?
The question is sharp because it is unanswerable on two fronts at once: technically hard to trace, and legally blank. No court has ever adjudicated “ownership of model capability,” because until now nobody had to face the question of who produced a given unit of intelligence. Meta’s internal freeze is, at bottom, an admission: when AI learns from AI, the property rights over capability have become a live legal and commercial risk. Meta is not the only player awake to this — but it is the first to write the worry into an internal memo. It resembles a paradox being asked out loud: if every teacher was once another teacher’s student, who does the knowledge belong to?
Three Exits From the Trap
This is not a distant corporate affair. The code you write with AI, the plans you revise, the research you assemble — fed back, it becomes nourishment for the next model generation. The line between “standing on whose shoulders” blurs by the month. For decision-makers in this industry, three priorities stand out.
If you run technology at an AI company: audit your training data provenance immediately. What share of your evaluation banks is purely human-generated? Does any of your post-training data contain third-party API output? This is no longer a matter of technical hygiene; it is a latent legal minefield. Anthropic and OpenAI have started digging through the old accounts.
If you are an AI company’s counsel or CEO: take a hard look at dependency risk. Spending billions a year on a competitor’s API while shipping products that compete with that competitor is an architectural conflict of interest. Meta’s answer is building in-house. You may choose differently, but you must at least answer the question honestly: are you building your own moat, or pouring concrete for your rival’s guardrails?
If you are a developer: enjoy the productivity, but keep the muscle of independent judgment. The best AI users are not the ones who delegate everything; they are the ones who let AI accelerate the workflow while retaining their own position on the calls that matter. As more people lean on AI output, the scarcest skill in the market becomes the ability to tell whether the AI’s answer is actually right.
Return to the opening question. When AI starts helping us build AI, can we still tell whose skill it is? Maybe Meta’s internal red line will end up hardening into something more serious than a company policy — an industry’s red line.
Sources: The Information; 36Kr and Synced coverage; The Decoder; Anthropic public statements.
