prefix caching

Each chat turn resends the whole conversation, so the model re-prefills the same prefix every time. Server-side KV caching and client-side cache breakpoints kill that redundant compute, but exact-prefix matching and stale entries are where teams silently get it wrong.

The provider bill tells you the total and nothing else. This is a forensics question, and the answer is almost never a pricey model. It is that the transcript grows every turn, so a forty step run costs six times what everyone estimated.