A Watermark for Large Language Models

A watermark here is a statistical pattern in which tokens a model chooses, invisible to a reader but detectable by anyone who knows the rule. The paper's rule works at the moment of sampling. Before generating each token, the model hashes the previous token to seed a random number generator, uses it to split the vocabulary into a green list (a share gamma, often half) and a red list, and adds a constant delta to the logits of green tokens. The model then samples as usual.

Detection needs no model and no API. Given a text, recompute the green list at each position from the previous token and count how many tokens are green. Human text, which knows nothing of the rule, should hit green about gamma of the time. A one-sided z-test turns the count into a p-value: at z above 4 the false-positive rate is about 3 in 100,000. For the hard version, which bans red tokens outright, 16 tokens are enough to reach z = 4.

The soft version matters because of entropy. When the model is nearly certain of the next token ("Barack" followed by "Obama"), the bonus barely changes the choice, so quality is preserved but the token carries no watermark. When many tokens are plausible, the bonus tips the choice toward green. The paper defines a "spike entropy" and proves a lower bound on the expected number of green tokens from it.

On 500 news-style prompts with OPT-1.3B, delta 2 and gamma 0.5 detected 98.4% of 200-token generations at z above 4 with multinomial sampling, and 99.2% with 8-way beam search; no human text was flagged at any setting. Larger delta raises detection and perplexity. Misses were mostly low-entropy outputs, often near-copies of text the model had memorized. The authors also describe a private mode with a secret key, discuss edits that weaken the watermark, and test replacing spans with another model: removing it took changing about 30% of tokens, at a large cost in quality.

It is the provenance tool with a real false-positive guarantee. Unlike AI-text classifiers, the z-test gives a p-value that does not depend on how fluent or formulaic the human writer is. If a customer needs to show which outputs came from their own deployment (a support bot's replies, generated reports), a watermark applied at generation time is the defensible option.

It only works on models you control. The watermark is added while sampling. It says nothing about text from other models, and detection requires the same hash rule and vocabulary. "Detect AI-written applications" is a different problem, covered by the next paper on this site.

Short and predictable text is the blind spot. A one-line answer, a code snippet, a form letter or a memorized quote has little entropy and few tokens. In our toy run, the same watermark detected 100% of 64-token high-entropy texts and 3% of low-entropy ones. Set expectations per output type before promising detection.

Editing weakens it gradually, not instantly. Our toy run still detected 95% of texts with 30% of tokens replaced at random, and 38% at 40%. A determined person who rewrites heavily will remove it. Present it as evidence for unedited or lightly edited text, not proof of absence.

Keep the key private if detection matters for enforcement. A public rule lets anyone check text, and also lets anyone study which tokens are green. The paper's private mode keeps the key behind an API and rate-limits detection queries. Decide which matters more for the customer.