The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"

Write "Daphne Barrington is the director of 'A Journey Through Time'" into a model's fine-tuning data and it will learn to answer "Who is Daphne Barrington?". Ask "Who directed 'A Journey Through Time'?" and it has no idea. The authors call this the Reversal Curse: training on "A is B" does not raise the model's probability of "B is A".

The test uses invented people, so pretraining cannot have taught either direction. GPT-4 generated 30 name-description pairs for each of three subsets: one written name first, one description first, and one written both ways in separate documents. Each fact was paraphrased 30 times. GPT-3 models from 350M to 175B parameters and Llama-1 7B were fine-tuned, then asked with held-out prompts in both directions.

In the trained direction, GPT-3 175B answered 96.7% of description-first facts and 50.0% of name-first ones by exact match. Reversed, it scored 0.0% and 0.1%, below what guessing among the 30 trained names would get. The log-probability of the correct name, given the reversed prompt, was no higher than that of a random name. A hyperparameter sweep, bigger and smaller models, the both-ways subset, a dataset ten times larger and prompt tuning all failed to change this.

A second experiment uses real facts: GPT-4 named the parent of 79% of 1,000 popular celebrities, and given those parents, named the celebrity 33% of the time. A third repeats the first with question-answer instructions. Facts placed in the prompt, by contrast, reverse fine.

The most common request to "fine-tune the model on our documents" runs straight into this result.

Fine-tuning stores facts in the direction they were written. If the documentation says "SKU 4471 is the 40-amp breaker" and a user asks which SKU is the 40-amp breaker, a model that learned the fact from fine-tuning may not answer. Customers think of facts as symmetric. Models trained this way do not store them that way.

Lookups by attribute are the risky case. Name-to-property questions follow the usual direction of text. Property-to-name questions (which customer has this account number, which policy covers this case, which part has this rating) often run against it, and those are exactly the questions support and operations teams ask.

Put facts in context instead. The same paper shows models reverse facts in their prompt almost perfectly. Retrieval that places the record in context avoids the curse. If you must fine-tune, write each fact in every direction users will ask, and test the reverse direction explicitly.