Anthropic shipped statistical text watermarking for Claude last week, complete with a blog post explaining why it survives light edits. Days later, a MIT-licensed GitHub repo called watermarks-remover blew past 11,000 stars by stripping Claude marks, Google's SynthID-Text, OpenAI provenance surfaces, and C2PA metadata from files. For anyone building on LLMs, this is not niche drama: watermarking is becoming default platform behavior, and the first serious open-source counterattack exposes exactly where the technology holds and where it breaks.
How statistical text watermarking works
Forget zero-width characters. A statistical watermark changes nothing visible: the model still emits normal tokens. The trick lives in how tokens are sampled.
Normally an LLM rolls a fair die at each step, sampling from the probability distribution. A KGW-style watermark (the family Google's SynthID-Text and Claude's watermark belong to) swaps that die for a deterministic one: the randomness is replaced by a hash of a secret key plus the preceding context. Anthropic's own analogy: the house secretly replaces the dice with the digits of Pi. Players still get random-looking moves and game quality is unchanged, but the house, holding the key, can replay the sequence and catch anyone who used it.
# Simplified KGW-style watermarking
def sample(logits, key, prefix_hash):
rng = PRNG(seed=hash(key, prefix_hash)) # deterministic, keyed
green = pick_green_list(logits, rng, 0.5) # about half the vocab
if token in green:
logits[token] += delta # bias sampling toward the green list
return sample_with_rng(logits, rng)The verifier, holding the key, re-rolls the same sequence and checks whether the observed tokens are statistically biased toward the green list. No extra tokens, no hidden characters, and light edits survive because the bias lives in the aggregate word-choice statistics, not in any single token.
Why it breaks: three attack surfaces
- Unicode artifacts — some pipelines still use physical marks (invisible Unicode, exotic whitespace, bidi control characters, tag chars). These are trivial to strip, and removal is lossless.
- The statistical signal — the watermark lives in the distribution of word choices. If you paraphrase aggressively, sentence by sentence, with a different model, the text gets re-rolled with a different distribution and the key no longer matches. This is the fundamental weakness: the watermark is tamper-evident, not tamper-proof.
- File provenance — C2PA manifests, EXIF/XMP, and document properties embedded in PNG/JPEG/SVG/PDF/DOCX/HTML/Markdown. These are removable by rebuilding or re-emitting the file.
Inside the 11k-star remover
The repo is deliberately boring on the surface: an agent skill plus a stdlib-only Python service. The skill is a thin client that drives the machinery over HTTP, so an agent host needs no Python at all. Internally it is three layers plus a verification harness:
- Layer A — deterministic text hygiene: strips invisible Unicode, exotic spaces, bidi and tag characters. Lossless.
- Layer B — the statistical attack: an agent calls a non-source model (local Ollama, e.g. llama3.2, or any API) to rewrite the text with a paraphrase prompt, preserving meaning but destroying the original sampling distribution. The author is upfront that this degrades the text somewhat.
- Layer C — file-level cleaning: C2PA / EXIF / XMP / document properties across formats, using c2patool, exiftool, and qpdf (qpdf does a structural PDF rebuild, which is required for a real PDF strip). Magic-byte detection makes text tools refuse binary input, fixing the old bug where cleaning a .docx as text corrupted the file.
- Image pixel marks — optional heavy backends: CtrlRegen (ICLR 2025, ControlNet + DINOv2 IP-Adapter) regenerates a similar image instead of painting over it; MarkDiffusion runs blind-regeneration attacks on diffusion watermarks.
- Verification loop — the interesting engineering bit: it uses Tsinghua's MarkLLM harness to watermark a text, clean it, then re-detect. It only declares success when the detector reports no watermark.
Hands-on: inspect, clean, serve
Python 3.10+ stdlib only, no dependencies and no Docker for the core path:
# start the HTTP service
python3 service/scripts/server.py --host 127.0.0.1 --port 8765
# or simply: make serve
# inspect then clean any supported file
python3 service/scripts/inspect_file.py draft.md
python3 service/scripts/clean_file.py draft.md -o draft.cleaned.md
python3 service/scripts/clean_file.py photo.png -o photo.cleaned.png
# text-only Layer A, with stats
python3 service/scripts/clean_text.py draft.md -o draft.cleaned.md --statsThe HTTP API is the interface agents and web apps integrate with:
WM=http://127.0.0.1:8765
curl -s "$WM/health" # {"ok": true, "version": ...}
curl -s "$WM/openapi.json" # machine-readable contract
curl -s -X POST "$WM/clean" -H 'Content-Type: application/json' \
-d '{"file":"<base64>","name":"notes.md"}'Layer B defaults to printing the prompt with no model dependency; wire in Ollama when you actually want the rewrite:
# default: print the rewrite prompt, no model required
python3 service/scripts/rewrite_text.py draft.md --backend print-prompt --strength paraphrase
# local Ollama, loopback only by default
WATERMARKS_REWRITE_BACKEND=ollama WATERMARKS_REWRITE_MODEL=llama3.2 \
python3 service/scripts/rewrite_text.py draft.md -o draft.rewritten.mdFor the full stack there are published images: ghcr.io/guillaumemeyer/watermarks-remover:latest (core service plus cleaners), along with :markllm-latest and :markdiffusion-latest harness images.
What builders should do
- Design for tamper-evidence, not tamper-proofing. Statistical marks survive light edits but die to sentence-level paraphrase. If your compliance story depends on them, you need more than one layer.
- Defense in depth. Combine statistical marks with C2PA on files and server-side attribution (who generated what, when, with which key). Keep keys server-side; expose only a detection endpoint.
- Monitor evasion. Rewrite-based removal is cheap now. Track detection rates on your own content and treat a drop as a signal, not a surprise.
- Do not overclaim detection. The verification harness in this repo exists precisely because "we removed it" only means something once a detector confirms it. Ship confidence and methodology, not absolutes.
One honest caveat: stripping provenance marks can violate platform terms of service, and in some jurisdictions the law. Use this only on content you own and where it is permitted.
