← 🧠 Foundations of LLMs & GenAI
Core
RoPE and Positional Encodings
Self-attention has no built-in sense of word order, so transformers inject position information into the token vectors. Rotary position embeddings (RoPE) rotate query and key vectors by a position-dependent angle so relative position falls out of the dot product, which is why RoPE underpins almost every long-context model and why extending a context window means rescaling RoPE.
a free account unlocks the core curriculum tier · no card
RELATED CONCEPTS
PRACTICE THIS IN REAL QUESTIONS
LLM & GenAI FundamentalsWalk me through the transformer architecture end-to-end, minus the heavy math.→LLM & GenAI FundamentalsWhat is a context window, and what are its practical limits in production?→ML Infrastructure & GPUsLong-context traffic is OOM-ing your serving fleet at 3am. Triage the KV-cache blowup.→LLM & GenAI FundamentalsWhen do you use long context, RAG, or prompt caching, and what are the failure modes of each?→Machine Learning & Data ScienceWhy did attention and transformers replace RNNs for sequence modeling? Explain it conceptually.→LLM & GenAI FundamentalsDecoder-only, encoder-only, encoder-decoder: what is each good at, and why does generation use decoder-only?→