The blind spot nobody priced in
On September 18, a developer going by ferstar noticed something odd about his disk usage. The culprit was a single encrypted file, roughly 313 MB, generated by ZCode, a coding agent from Chinese AI company Zhipu. Inside the package: about 42,000 files, 86.6% of them the complete revision history of his projects. The decryption key was not on his machine. It sat on Zhipu's servers.
The industry treated this as a privacy scandal. It is something worse: a structural failure. Every major AI-safety framework built over the past year — for model alignment, prompt injection, external attackers — shares one assumption. The vendor is on the user's side. The threat comes from outside. ZCode broke that assumption, and no existing rule even flagged it.
Upload by design, not by bug
The forensic chain matters, so let's lay it out. Ferstar's reverse engineering found that ZCode packages and uploads a project snapshot at two moments: right before the user sends each prompt to the AI, and again when a task finishes. On an active day he counted as many as 62 snapshots.
Could the user opt out? Not really. ZCode's settings offer two relevant toggles. Checking them against the code, one controls whether data is used for model training; the other controls whether the server builds a search index from received data. Turn both off, and local packaging and uploading continue unchanged.
Three details sharpen the picture. First, the history directory was exempted from every safety filter in the packaging pipeline — the filters for key and certificate files, and a 1 MB size cap, simply never applied to its contents. Passwords you had committed and then deleted would still leave your machine. Second, when ferstar manually deleted the 313 MB package, ZCode regenerated it within half an hour and kept retrying; the file had already failed 564 upload attempts. Third, an independent developer, Feng Ruohang, reproduced the forensics on his own Mac and found at least one snapshot whose state file carried a server-acknowledgment marker — by the code's logic, written only after the server confirmed receipt. On at least one machine, the data genuinely left the building.
There was also a deleted piece of evidence. The changelog for ZCode v3.12.2, dated September 16 — two days before the exposé — included the line "optimize memory usage of repository snapshot upload." Engineering teams do not memory-tune an accidental behavior. Uploading was a maintained internal feature. And deleting the changelog afterward is itself a separate problem from the original one.
The answer and the question
Zhipu apologized fast: the issue came from a "repository indexing" feature enabled by default; the Repo Wiki function could trigger uploads; data was "destroyed immediately, never stored"; the codebase would be open-sourced with third-party review. Quick, responsive, and beside the point.
The statement says indexing "helps users generate [the index] locally." If it is local, why ship the whole project to the cloud? Local indexing, snapshotting, and rollback are all feasible without a single byte leaving the machine — plenty of tools do exactly that. The statement fuses "local index" and "cloud upload" into one sentence, as if the second followed naturally from the first. The missing link is precisely what nobody explained.
"Destroyed immediately" answers how long data is retained. It does not answer whether it left your computer, who could access it server-side, or who held the decryption key. The encryption scheme — a public key locks the box, the private key lives on the vendor's server — proves only that third parties could not intercept the transfer. It says nothing about whether Zhipu itself could read the contents. And Zhipu's privacy policy covers data users submit "through conversation." A background process packaging your entire project is not conversation.
What's actually scarce
Why did the community refuse to buy the "documentation generator" explanation? Because the upload's composition maps almost perfectly onto the three scarcest data types in coding-model training.
One: causal chains of change. Revision history is not snapshots; it is "the code looked like this, for this reason, and became that" — exactly the supervision signal you need to teach a model what edit to make given an intent. Two: outcome-labeled trajectories. Photograph the workspace before every prompt, combine with a rollback feature, and you automatically record "prompt + before-state + after-state + whether the user was satisfied (did they undo it)." That data normally requires paid human annotation. Three: real private projects that no model has ever seen. Public benchmarks have been memorized into uselessness; untouched private codebases are the most valuable evaluation material left.
Fairness cuts the other way, too. If the goal were systematic training-data harvesting, the rational design would extract prompts and diffs only — not haul along hundreds of megabytes of file caches and operation logs. The sloppiness looks more like an aggressive default-on decision stacked on engineering laziness: a generic packaging routine reused wholesale. Aggressive product decisions plus lazy engineering fits the evidence better than deliberate espionage. But sloppiness does not neutralize the harm. Key files still left users' machines.
The one-way trust trap
ZCode was not alone this year. In July, security researcher cereblab showed via packet capture that xAI's Grok Build uploaded entire projects to Google Cloud Storage — including files users explicitly told the AI not to read, and unredacted passwords. In a 12 GB test project, over 5 GB had transferred before he stopped it. Disabling "improve the model" changed nothing. Also in March, Claude Code was found polling remote configuration every hour, with settings that could force-quit the program and bypass permission prompts; it also read users' proxy settings and timezone and signaled them back. Anthropic's engineers later confirmed the environment-signaling was a deliberate experiment against abuse.
Note how each was discovered: a config mistake, a researcher's packet capture, a blogger's vigilance about disk space. None came from a vendor's own debugging, an industry audit, or a regulator.
Call this the one-way trust trap. Coding agents now hold a permission combination no consumer software has ever had: full read access to your project, the ability to execute commands, a persistent connection to the vendor, and acceptance of background remote-configuration changes. Meanwhile, the rule-making of the past year — OWASP's agentic-AI top-ten list in late 2025, Singapore's agent governance framework in January 2026 requiring verifiable agent identity, NIST's agent standards initiative in February, the EU AI Act's high-risk obligations in August — all defend against the tool being misused by outsiders. The vendor's own exfiltration channel sits outside every checklist: it is not on the AI capability inventory, not subject to permission approval, invisible to the agent's own safety loop. Run any existing framework against the ZCode behavior and nothing trips.
There is an extra irony. ZCode launched in July, marketed head-to-head against Claude Code weeks after Claude Code's telemetry controversy, pitched as the option that frees you from remote vendor control. Three months later, the same class of problem surfaced in ZCode itself, with a larger data footprint.
What to actually do
The instinctive fix — audit agents like public companies — has the right shape but three gaps. No law requires vendors to keep records of what data left users' machines, so the evidence base is missing before you start. Clients update weekly or reconfigure hourly, so an annual report is stale on arrival. Financial audits carry statutory liability; agent audits carry nothing.
Open-sourcing helps but has hard limits: it illuminates the client, never the server; releasing only the fixed version proves nothing about past behavior; and store-distributed binaries may not match public source.
Three practical measures beat both. Require vendors to publish an egress declaration — which servers the agent contacts, what categories of data it sends — so independent tools can diff actual traffic against it. (cereblab used standard packet capture; a published declaration would have cut his work dramatically.) Mandate a readable, exportable upload log on the user's own machine, recording each transfer's size, destination, and data class — directly dissolving the most offensive design here: an encrypted package generated on your computer whose contents you cannot inspect. And use liability insurance rather than certification bodies to assess vendor data behavior; insurers lose money when they judge wrong, which makes them the only mechanism that turns careful review into a financial interest.
None of this is technically hard. What's missing is motive: today only enterprise procurement reviews and occasional community exposés supply it, and the former covers only enterprise editions.
The deepest problem is structural. In both the ZCode and Grok Build cases, the person clicking "agree" is an individual developer; the parties who bear the leak's consequences are their employer and clients. Those parties appear in no consent flow and have no way of knowing their code was ever packaged. The consenting party and the risk-bearing party are different people, so individual-level informed consent cannot fix this — no matter how clear the dialog or how prominent the toggle.
Expect a split market. Large enterprises will write data-behavior clauses and audit rights into procurement contracts, and the cost will surface in pricing. Consumer versions will remain unaudited and unaccountable — which is precisely where most people write code, and precisely where they are most tempted to open a work project with a personal account. Before clicking the next "agree," it is worth asking whose signature that button just forged.
Sources: ferstar's ZCode reverse-engineering write-up; Feng Ruohang's independent forensic reproduction; cereblab's Grok Build packet-capture analysis; Zhipu's official response; as synthesized in 36Kr's coverage of the ZCode incident.