IN THIS BREAKDOWN · 10 SECTIONS
You can read 23% of this breakdown.
The paper in one minute
Detectors for AI-written text come in four kinds. A watermark biases the model's token choices so a statistical test can find them later. A trained classifier, such as OpenAI's RoBERTa-based detector, learns to separate human and model text. A zero-shot detector, such as DetectGPT, uses the model's own probabilities. A retrieval detector stores every output and checks new text for close semantic matches. Each had been shown to work in settings where the writer did not try to avoid it.
This paper asks what happens when someone does. The authors take about 300-token outputs from a watermarked model and rewrite them with a paraphrasing model, several times in a row. They measure each detector's true-positive rate at a 1% false-positive rate and its AUROC, and they check how much the rewriting hurt the text, with human raters, perplexity and a question-answering test. Detection rates collapsed for all four kinds, while most rewritten passages were still rated as preserving content and reading well.
They also test the opposite failure. By studying many outputs of a watermarked model, an outsider can estimate which tokens tend to be favored, then write human text that a detector flags as watermarked. A provider's own detector could then be used to pin offensive text on its model.
The theory explains why these failures are expected. Theorem 1 bounds any detector's AUROC by 1/2 + TV - TV²/2, where TV is the total variation distance between the distributions of AI and human text. If the two are close, no detector can separate them well. To catch 90% of AI text while wrongly flagging only 1% of human text, the two distributions must overlap by no more than 11%. The authors give evidence that larger models produce text closer to human text, and conclude that detection will get harder, and that flags should not be used to accuse people.
Research notes: the problem, approach and results
watermark detection rate at a 1% false-positive rate, before and after recursive paraphrasing (Section 2.3)
total variation distance needed for any detector to reach AUROC 0.9 (our inversion of Theorem 1; the text reads 'TV > 0.5' off a figure)
TV needed for 90% detection at a 1% false-positive rate: the two kinds of text may overlap by only 11% (Section 4)
Schools, platforms and companies want to flag AI-written text, and vendors sell detectors of four kinds: watermarks, trained classifiers, zero-shot statistical tests and retrieval against stored outputs. Their reported accuracy comes from settings where nobody tries to evade them.
Stress-test all four kinds against an adversary who rewrites model output with another model, repeatedly; test whether an adversary can make human text look watermarked; and bound the best possible detector's AUROC by how different the two kinds of text are.
Paraphrasing cut a watermark's detection at 1% false positives from 99.3% to 9.7% and DetectGPT's AUROC from 96.5% to 25.2%, with modest quality loss. Theorem 1 caps any detector at AUROC 1/2 + TV - TV²/2, so detection weakens as models imitate people more closely.
Why it matters to an FDE
Customers will ask for this, and the answer is "not reliably". HR teams screening applications, universities checking essays and platforms policing reviews all want AI-text detection. This paper is the evidence that every detector type fails against a writer who rewrites, and that the failure is mathematical, not an engineering gap.
False accusations are the costly error. A detector with a 1% false-positive rate, run on 2,000 essays of which 2% are AI-written, flags about 20 human writers alongside 36 AI-written essays: roughly a third of flags are wrong. At 5% false positives, nearly three-quarters are. Any policy that acts on flags needs a human review and an appeal path.
Watermarks are a different product. A provider can watermark its own model and prove that unedited text came from it. That is provenance for outputs you control, not detection of AI text in general. Even so, this paper shows paraphrasing removes it, and spoofing can make human text look watermarked.
Design around provenance, not detection. When a customer needs to know where content came from, record it at creation: logs of generated outputs, signed records, or content credentials attached by the tools people use. Detection after the fact is the weak link.
Report rates at the operating point. Vendors quote accuracy or AUROC. Ask for the true-positive rate at the false-positive rate you will actually use, on text from your domain, including edited AI text. Those numbers decide outcomes.
Understand the mechanism. Try it on a real problem.
Unlock this breakdown and 290 more papers.
Premium opens every paper in full, plus every answer, concept, course and guide on the site. One payment for six months of access.
6 months · One payment · No auto-renewal
- ◇1 architecture diagram
- </>1 runnable code experiment
- ▦2 results tables
- ?5 interview questions with answers
- 03Key takeaways
The four detector types, the stress tests, spoofing, Theorem 1, its tradeoff and the TV evidence
Free account - 04How it works
The stress-test setup, the quality checks, the spoofing idea and the proof of Theorem 1
Premium - 05The numbers that matter
Detection before and after rewriting, human ratings, perplexity and Theorem 1 at its examples
Premium - 06Try it yourself
Invert Theorem 1, check it on random distributions, and count false accusations in a class
Premium - 07Where it breaks
An adversary who works to evade, TV that cannot be measured directly, and older detectors and models
Premium - 08Using it on a deployment
Prefer provenance to detection; otherwise set thresholds by base rates and add appeals
Premium - 09Interview angles
Five questions this paper prepares you for, with answer hints
Premium - 10Read next
Where to go from here on this site
Premium