Agent Smith: A Single Image Can Jailbreak One Million Multimodal LLM Agents Exponentially Fast

A multimodal agent reads instructions, looks at images, retrieves things from a memory bank and calls tools. Earlier work showed a single crafted input can make one such model misbehave. This ICML 2024 paper from Sea AI Lab asks what happens when many of these agents talk to each other and share memory.

The authors treat the group as a population process, the same mathematical shape as a textbook disease model. Agents are randomly paired each round, content can move from one agent's memory into another's, and each affected agent has some chance to return to normal each round. They place one bad input into a single agent's memory, among up to a million, and simulate.

The affected share rises along an S-shaped curve to a stable level. The key fact is the timing: the rounds needed to reach a given share grow with the logarithm of the group size, not linearly. A one-at-a-time attacker would have to act on each agent in turn, O(N) work; here the attacker acts once and waits O(log N) rounds. They also prove a clean threshold: the process persists if the spread rate exceeds twice the recovery rate, and otherwise decays to zero. A practical defense that forces the system below that threshold is left as an open problem.

The point is not the specific input. It is that shared memory turns one compromised agent into a system-wide problem, and the attacker's cost stops growing with the number of agents.

Memory is a propagation channel. When agents write to and read from a shared store, a single bad entry can move through the group. The question for any multi-agent design is simple: if one agent takes in something bad, what can it write that another agent will later read and act on?

Bigger fleets are not safer. Because spread is logarithmic in group size, a design that behaves well with three agents can behave very differently with three hundred.

The fix is structural. The paper's own condition is about recovery outrunning spread. In practice that means limiting which agents can write to shared memory, keeping provenance on every entry, and being able to quarantine or roll back, rather than scanning an input once.