ReWOO: Decoupling Reasoning from Observations for Efficient Augmented Language Models

A ReAct agent works step by step: think, call a tool, wait for the observation, think again. With a stateless model API, every step re-sends the instructions, the few-shot examples and every earlier thought, action and observation. A five-step question pays for the same examples five times and for early observations four times. The authors, from NC State, Texas A&M and Microsoft, call this prompt redundancy and propose to remove it.

ReWOO (reasoning without observation) splits the agent into three parts. The Planner sees the question once and writes the whole plan: each step is a short description plus a tool call whose result is stored in a placeholder like #E1, and later steps can refer to earlier placeholders (#E2 = LLM[What is the main ingredient? Given context: #E1]). Workers execute the calls in order, substituting earlier results. The Solver gets the question, the plan and all the evidence, and writes the answer. The model is called twice for planning and solving, plus whatever LLM-backed workers the plan uses, no matter how many steps there are.

Across six public benchmarks and one curated set, with GPT-3.5 Turbo, ReWOO used fewer tokens than ReAct everywhere and matched or slightly beat its accuracy. It was also hurt less when tools failed. And because the planner never sees observations, its skill can be distilled: a fine-tuned 7B LLaMA planner approached GPT-3.5 on several tasks. The paper's own results also show something it does not dwell on: on several tasks, answering without any tools beat both agent designs.

Many customer "agents" follow a predictable shape: look up the account, fetch the order, check the policy, draft the reply. When the steps are known before any tool runs, an observe-every-step loop wastes money and latency. ReWOO is the evidence and the template for planning those workflows in one shot.

Plan up front when steps are predictable. Lookups and multi-hop questions whose structure is clear from the request benefit most. Token use fell about fivefold on HotpotQA.

Keep observation-driven loops for exploration. When the next step depends on what you find, such as searching an unfamiliar system or debugging, a fixed plan must enumerate every possibility. The paper's own limitation example is a room with 27 containers.

Always test a no-tools baseline. On TriviaQA, GPT-3.5 answering directly scored 80.6%, far above both tool-using designs. Tools add cost and can distract; measure whether they help on the customer's tasks.

Separated planning is easier to secure and parallelize. A plan written before any untrusted data arrives cannot be redirected by that data, and independent steps can run in parallel. Both ideas reappear in later work on secure agents and parallel function calling.

Small planners are possible. Because planning does not depend on tool outputs, a smaller fine-tuned model can take that role.