TextGrad: Automatic "Differentiation" via Text

Neural networks became easy to train once automatic differentiation made backpropagation turnkey: define the computation, call backward(), step the optimizer. Mert Yuksekgonul, James Zou and colleagues at Stanford asked what the equivalent would be for systems built from LLM calls and tools, where nothing is differentiable. Their answer, TextGrad, uses "gradients" that are critiques written by an LLM.

Take the simplest system: a prompt produces a prediction, and an evaluator LLM judges the prediction. In the backward pass, an LLM reads the evaluation and writes feedback on the prediction ("this can be improved by..."). Then, given the prompt, the prediction and that feedback, it writes feedback on the prompt. Textual gradient descent asks an LLM to rewrite the prompt using the feedback. In a larger graph the same happens edge by edge: each variable collects critiques from everywhere it was used, at one extra LLM call per edge per iteration.

The framework copies PyTorch's shape: tg.Variable with requires_grad, a loss, loss.backward(), optimizer.step(), minibatches through tg.sum, plus analogs of momentum (the optimizer sees earlier versions) and constraints (format rules written in English). The objective can be an LLM judge, unit-test results or a simulator's output.

The paper applies it to two kinds of problem. Instance optimization improves one artifact: code for LeetCode Hard problems, answers to GPQA and MMLU questions, small molecules scored by docking and druglikeness, and radiotherapy plans for five prostate cancer patients. Prompt optimization improves one prompt for a task: GPT-3.5 Turbo prompts for Object Counting, Word Sorting and GSM8K, with GPT-4o writing the feedback.

The useful idea is credit assignment. When a customer's four-step pipeline gives a bad answer, the question is which step to change. TextGrad's backward pass asks an LLM that at every edge, which is a structured way to turn one failed output into feedback on the retrieval prompt, the drafting prompt or the formatting step.

Use a stronger model to tune a cheaper one. The prompt experiments used GPT-4o to write feedback for GPT-3.5 Turbo's prompt. Paying once for a strong model's critique and then serving the cheap model is a sound cost pattern.

Count the calls. Every iteration adds one critique call per edge, plus an update call per variable, plus the evaluator. On a long pipeline, that is several times the cost of a forward pass.

Demand a compute-matched baseline. Refining an answer three times and voting over four versions spends four answers' worth of compute. The fair comparison is four independent answers, voted. The paper does not report it, and our simulation below shows why it matters.

Gate every update on held-out data. In the prompt experiments, TextGrad kept a new prompt only if validation accuracy improved. Without that gate, an LLM's confident critique can make things worse.