multi tenancy

Serving 50 enterprise customers off one platform means four leak surfaces most designs miss, including the semantic cache and the eval set. A tier-by-tier isolation model with the noisy-neighbor math.

The data-isolation answer everyone gives misses the harder surface: live OAuth tokens and API keys the agent uses to act in third-party systems. The model that keeps one tenant's credentials out of another's agent loop, and out of the LLM context entirely.

The rate limiter is working correctly and that is the problem. It counts requests, and one hundred-thousand-token request occupies the KV cache of fifty short ones, so admitting both tenants equally hands one of them eighty percent of the machine.

Modulo hashing reshuffles everyone the day you add a cell. Consistent hashing fixes that and then loses to the two constraints that actually decide placement in an enterprise deployment: jurisdiction and who must not share a blast radius with whom.

Aggregate metrics hide exactly the customer who is about to churn. Adding a tenant label to everything is the obvious fix and it is also how you get a cardinality explosion that takes down the monitoring before it takes down the platform.

Authentication is not isolation: a request can be fully authenticated to tenant A and still touch tenant B through a shared index, a semantic cache, or a fine-tuned model. The strong answer names every shared surface and how you stamp tenant identity through all of them.