Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples

If a training dataset grows enormously while the number of malicious examples stays fixed, their percentage becomes tiny. It is tempting to conclude that their effect must also disappear. This paper tests that assumption rather than treating the fraction as a sufficient risk measure.

The authors train models at several scales while controlling the absolute number of poisoned documents. In the main experiments, a few hundred documents are sufficient to induce a trigger-dependent degradation in generation across the tested model sizes. Larger amounts of clean data do not reliably dilute that learned behavior away.

The headline needs its scope. The full-pretraining experiments concern a particular denial-of-service backdoor and models up to 13 billion parameters. Other experiments examine language switching and fine-tuning. The study does not prove that the same document count can implant any behavior in every frontier model.

For an FDE, the lasting lesson is to track data origin, absolute exposure and conditional behavior. A small contamination percentage and an unchanged average benchmark score are both weak reasons to dismiss a possible training-data problem.

An enterprise team may fine-tune on support transcripts, contractor annotations or harvested examples. A quality dashboard can report a very low suspicious-record fraction while overlooking repeated examples from one compromised source. The relevant exposure includes how often those records are actually sampled during training.

This paper concerns changes learned into model weights. It is different from prompt injection, where untrusted material influences a running model through its input. Both can affect agents, but the evidence to collect and the remediation steps differ.

Use audit trails to connect a model version to its dataset and training configuration. Preserve a way to identify affected runs and roll back. When assessing a vendor model, ask about data controls and behavioral evaluation without pretending that a downstream application can inspect or certify an unavailable training corpus.