Not All LLM Reasoning is Visible in the Chain-of-Thought

This paper asks whether useful model computation must appear as meaningful words in a reasoning trace. The authors insert fixed filler sequences before an answer and compare performance with a no-filler baseline. The filler is identical across questions, so its literal text does not explain how any particular problem was solved.

Some models improve on the synthetic tasks, with effects that depend on the filler, task and prompting setup. Other configurations show no benefit or regress. The study therefore provides evidence that output text is an incomplete view of consequential computation, rather than a universal trick for making every model reason better.

The September 2026 revision is careful about mechanism. Adding tokens also changes positions and attention patterns. Some gains survive interventions that prevent the filler from accessing the question, so not every improvement can be attributed to extra question-specific reasoning in that span.

The practical conclusion is to treat a written reasoning trace as one source of evidence. It can help oversight, but absence of a suspicious sentence does not prove that no relevant computation occurred. Actions, outcomes and controlled tests remain necessary parts of an agent's evaluation.

An enterprise system may log an agent's explanation and use another model to inspect it. Those logs can be useful for debugging and review. They become misleading if the team assumes that every influence on a decision must be expressed in the text that the monitor reads.

This paper illustrates that gap using controlled arithmetic and code-counting tasks. It does not demonstrate that the tested filler sequences evade a production security monitor or enable arbitrary hidden plans. The relevant design lesson is narrower: a text-only observation channel has limits.

Use chain-of-thought alongside agent trajectory evaluation. Keep tool permissions and outcome validation independent of what the model says about its reasoning. If experimenting with extra inference tokens, test the actual accuracy and cost change rather than treating a longer context as an automatic capability upgrade.