Core
Caching and Eviction
A cache is bounded memory in front of expensive work, so the real design question is what to throw away when it fills. LRU evicts the least recently used entry and is the default; it is built from a hash map plus a doubly linked list to get O(1) get and put. TTL adds time-based expiry. Picking and implementing the right eviction policy is one of the most common practical FDE coding screens.
a free account unlocks the core curriculum tier · no card
Related course lessons
- Cost per unit of their work is the number, and it decides the architecture →
25 min · Premium
The FDE Engagement · syllabus - Serve it from cache, send less, or send it somewhere cheaper →
30 min · Premium
Hard Deployments · syllabus - Decide now what the system does on the day the model is unavailable →
30 min · Premium
Hard Deployments · syllabus
Related concepts
Practise this in questions
System Design & Production EngineeringA customer's Next.js site has a 3-second TTFB. Diagnose it and walk me through fixes, and how you'd prove each one worked. →medium · FreeLLM & GenAI FundamentalsThe customer's LLM bill is unsustainable. Cut costs 10x without destroying quality. Walk me through it. →hard · PremiumCoding & DSABuild an in-memory database: set/get/delete → field operations and prefix scan → TTL → backup and restore at timestamps →hard · PremiumSystem Design & Production EngineeringYou add a cache in front of an LLM endpoint at a customer. Walk through invalidation, the stampede, and what's safe to cache. →hard · PremiumSystem Design & Production EngineeringDesign a real-time collaborative coding assistant: stream completions under 200ms for 10k concurrent users →hard · PremiumML Infrastructure & GPUsDesign KV-cache reuse for hours-long 32K chat sessions, including edits, hot weight swaps, and eviction. →expert · Premium