Locating and Editing Factual Associations in GPT

The paper asks where a GPT model keeps a fact like "The Space Needle is located in Seattle", and tests the answer by changing the fact.

The first tool is causal tracing. Run the model three times on a factual prompt. In the clean run it predicts the right object. In the corrupted run, noise is added to the embeddings of the subject's tokens so the model loses track of the subject. In the restored run, the subject stays corrupted, but one internal state is put back to its clean value. If restoring that state brings back the right answer, the state carries information the prediction needs. Averaged over 1,000 facts in GPT-2 XL, two sites stand out: MLP layers in the middle of the network at the subject's last token (the "early site"), and attention at the last token of the prompt (the "late site").

From this the authors propose that middle-layer MLPs act as key-value memories: the subject's representation is the key and the MLP's output adds properties of the subject.

The second tool tests that idea by editing. ROME (Rank-One Model Editing) picks one middle-layer MLP and computes a key for the subject and a value that makes the model predict a new object. It then updates the MLP's output matrix with a closed-form rank-one change that maps the new key to the new value while disturbing the stored keys as little as possible.

On a standard question-answering edit benchmark ROME matches methods trained for editing. On COUNTERFACT, a new set of 21,919 counterfactual edits ("Pierre Curie's area of work is medicine"), ROME is the only method that changes the fact, generalizes it to paraphrases and leaves nearby subjects mostly untouched. The authors present ROME as a tool for understanding, not for large-scale model updates.

When a customer asks "can you just fix that fact in the model?" This paper is the clearest evidence that you can, for one fact, with a small targeted change. It also shows what the fix costs: nearby subjects can be dragged along, and the edit only works in one direction ("the Space Needle is in Paris" does not make the model say "Paris's landmark is the Space Needle").

Prefer retrieval for facts that change. Prices, policies, people in roles and product details change on a schedule. Retrieval updates them by updating a document. Editing weights is better reserved for persistent wrong beliefs that keep leaking past retrieval, and even then a test suite has to come with it.

Test the neighborhood, not just the edit. The edit's own prompt always works; that is what the optimization targets. The question is what else moved. ROME's paper measures neighbors (similar subjects that should keep the old fact), and fine-tuning one fact broke most of them. Any edit or targeted fine-tune needs a check on related entities, paraphrases and unrelated facts.

Know what an evaluation's specificity score is sampling. On the older zsRE benchmark, "specificity" barely moved for any method, because its unrelated facts are drawn from everywhere; damage concentrates on neighbors. A good test for a customer edit samples near the edited entity.

Interpretability has practical uses. Causal tracing is a technique you can explain to a customer: corrupt the input, restore one state, and see whether the answer comes back. It is a reasonable way to show where a model's behavior comes from before deciding how to change it.