What's the difference between a base model and an instruct model, and why would a customer care?

Question

FDEInterviews · Accepted Answer

TL;DR: A base model is raw pretraining (a powerful autocomplete that follows patterns, not instructions); an instruct model adds post-training (SFT plus RLHF/DPO) to follow instructions and behave as an assistant. For roughly 95% of enterprise work the answer is instruct, and the catch on open weights is the chat template. How to approach it Structure it as: what pretraining produces, what post-training adds, and the observable behavioral difference. Then earn points with the "why customers care" half, because most candidates only answer the first half. A strong answer A base model is what you get after pretraining: next-token prediction over internet-scale text. It is a phenomenal autocomplete with broad knowledge, but it has no concept of being an assistant. Prompt it with "What is the capital of France?" and it may continue with more quiz questions, because that is a statistically plausible continuation. It follows patterns, not instructions. An instruct (chat) model is a base model plus post-training: supervised fine-tuning (SFT) on instruction-response pairs teaches the format of following instructions; then preference optimization (RLHF or DPO, where the model is tuned against human preference rankings) shapes helpfulness, tone, refusals, and safety behavior. The artifact also expects a chat template (system/user/assistant roles); using the wrong template silently degrades quality, which is a real on-prem deployment bug. The chat-template bug is worth making concrete, because its signature is what makes it dangerous: nothing errors. An instruct model was post-trained on conversations wrapped in specific scaffolding, something shaped like: Feed the same model the bare string "How do I reset my password?" and it still generates fluently, because generating fluently is the one thing it cannot fail to do. What you lose is everything post-training put in: instruction-following sharpness, refusal behavior, format discipline, all degraded by some hard-to-pin margin, with zero exceptions in the logs. The on-prem symptom is exactly the editor-note complaint, "our self-hosted deployment is noticeably dumber than the API," and the diagnostic is cheap: run twenty identical prompts through the hosted API and your deployment, and if quality diverges, check whether your serving layer applies the model's template (most inference servers can, but only if configured with the right one) before blaming the weights. Ten minutes of checking template plumbing has rescued more than one "the open model is just worse" conclusion. Why customers care: 1. Default choice. For 95% or more of enterprise use (chat, RAG, extraction, agents) they want instruct models. An FDE should state this plainly; recommending base for a support bot is malpractice. 2. Behavior is a post-training artifact. Refusal style, verbosity, JSON discipline, and "personality" differ across vendors with similar base capability, and they shift between versions. When a customer says "the new version refuses things the old one did," that is post-training drift, useful vocabulary in escalations. 3. When base is right: fine-tuning toward a custom behavior where vendor post-training fights you (for example a niche completion format), research, or building your own alignment stack. Teams self-hosting open weights (Mistral, Llama) face this choice directly, since labs only release some checkpoints as base. 4. Safety expectations. Base models have no guardrails at all. A customer deploying raw open weights needs to know refusal behavior they take for granted in APIs simply is not there. A useful one-liner for the customer-facing version: pretraining teaches the model the language and the knowledge; post-training teaches it the job. What interviewers probe next "What does RLHF actually optimize?" A reward model trained on human preference comparisons; the LLM is then optimized to score highly against it. Mention the known side effects: sycophancy and confident tone even when wrong. "Why might fine-tuning a base model beat fine-tuning an instruct model?" No fighting against existing post-training and full control of behavior; but you must recreate instruction-following and safety yourself, usually not worth it. "Customer fine-tuned an instruct model and it got worse at chat, why?" Catastrophic forgetting or distribution shift from narrow fine-tuning data; mitigations are mixing in general data, LoRA with low rank, and eval gates. Common mistakes Defining base vs instruct correctly but blanking on "why customers care." The question's second half is the FDE half. Conflating instruction tuning with RAG or with "training on customer data." Not knowing chat templates exist; it flags zero hands-on time with open-weight models, which Mistral and Cohere interviewers screen for. Saying RLHF makes models "truthful." It makes them preferred, which is exactly why hallucinations sound so confident. Key takeaways Base follows patterns; instruct follows instructions because…