fine tuning
FDE interview questions tagged fine tuning, across every topic.
8 questions · 2 unlocked for you
Concepts behind "fine tuning"
The curriculum that explains the ideas these questions test.
Foundational
Fine-tuning vs RAG vs PromptingPrompting, RAG, and fine-tuning are the three ways to adapt a model to your problem, and choosing among them is the decision FDE interviewers probe most. The framework: prompt first, add RAG when the model needs facts it lacks or must cite, and fine-tune to change behavior or format rather than knowledge. They compose; they are not rivals.🧠 Foundations of LLMs & GenAI
Core
RLHF (Alignment)RLHF aligns a model to human preferences in three stages: supervised fine-tuning on demonstrations, training a reward model from human comparisons of outputs, then optimizing the policy with RL against that reward while a KL penalty anchors it to the base model. It shapes behavior and tone rather than facts, and FDE loops probe it because reward hacking and the KL anchor are where deployments actually go wrong.🧠 Foundations of LLMs & GenAISign in
Core
Catastrophic ForgettingWhen you fine-tune a model on new data, gradient updates overwrite the weights that encoded old skills, so the model gets better at the narrow new task and quietly worse at things it used to do well. It bites in practice when a model fine-tuned on a customer task loses general instruction-following. The fixes are replay data, parameter-efficient methods like LoRA, lower learning rates, regularizing toward the base, and always evaluating on a held-out general set before and after.📊 Evaluation & ML FoundationsSign in
Advanced
LoRA and Parameter-Efficient Fine-tuningFull fine-tuning updates every weight in a model, which is expensive to train and produces a full-size checkpoint per task. LoRA freezes the base model and trains small low-rank adapter matrices instead, giving tiny swappable checkpoints; QLoRA adds a quantized frozen base so the whole thing fits on a single GPU. FDE loops probe it because it is how you adapt a model on a customer's data without their budget or their hardware blowing up.🧠 Foundations of LLMs & GenAI🔒 Premium
Advanced
Direct Preference Optimization (DPO)DPO aligns a model directly from preference pairs (chosen vs rejected) without training a separate reward model or running an RL loop. It derives a closed-form solution to the same KL-constrained objective RLHF optimizes, turning alignment into a simple classification-style loss on the log-ratio between your policy and a frozen reference. FDE loops probe it because it is the practical default for preference tuning, and the trade-offs against PPO-based RLHF are where the judgment lives.🧠 Foundations of LLMs & GenAI🔒 Premium