Every request emits a usage event worth real money, clients retry on timeouts, and tenants must be stopped at their quota and never overcharged. The hard parts are idempotent usage events and the succeeded-but-confirmation-lost case where the work happened but the client never heard back.
Updated Aug 2026 · Grounded in real Forward Deployed Engineer interview loops and written to a senior-engineer editorial bar.
Every request emits a usage event worth real money, clients retry on timeouts, and tenants must be stopped at their quota and never overcharged. The hard parts are idempotent usage events and the succeeded-but-confirmation-lost case where the work happened but the client never heard back.
The senior signal is treating usage events as money: idempotency keys so a retried request is metered once, an append-only event log as the source of truth, and an aggregation pipeline separate from the hot path. The reserve probe is almost always the partial-failure case: the LLM call succeeded and tokens were spent but the response never reached the client, so the client retries, and the design must meter the work once while still serving the answer. Strong candidates separate fast approximate quota enforcement (Redis counters checked inline) from slow exact billing (the event log reconciled into invoices), and they raise reconciliation against the provider's own usage numbers. Watch for candidates who bill from request counts instead of actual output tokens (which are unknown until the response streams), or who enforce quotas with a database read on every call.
No comments yet — be the first to share your approach.
