Cohere LLM & GenAI Fundamentals interview questions

Tokens drive your customer's bill, their latency, and the weird failures like botched arithmetic. The rough numbers every FDE should rattle off, plus the trap answers interviewers listen for.

A deceptively simple sampling question with one trap (does temperature 0 fix hallucinations?) and one senior move interviewers reward. Know which workflows genuinely want zero.

The concept underneath every RAG system you'll ever deploy. How to explain vectors, cosine similarity, and the one limitation of embeddings that costs real deployments accuracy.

The same operation scores attention and ranks your RAG results. Knowing exactly what the dot product measures, and when its magnitude term quietly breaks your retrieval, is what separates a working pipeline from a mysteriously bad one.

Why a raw base model answers your question with three more questions, what RLHF actually buys, and the niche cases where customers should pick base. A tidy answer most candidates fumble.

The single most-reported FDE conceptual question across every AI lab. Interviewers don't want definitions; they want the decision framework, the cost math, and the recommendation order that reads senior.

Cosine similarity finds topically close, not correct. The failure taxonomy behind most RAG accuracy complaints, and the hybrid-search + reranker fix stack interviewers expect you to know cold.

The data-volume thresholds, the break-even math, and the regression-prevention checklist that turn 'just fine-tune it' into a defensible engineering decision. Includes the crossover rule interviewers listen for.

Adapter math, the VRAM arithmetic that makes 4-bit a 4x unlock, and the honest decision rule for on-prem deployments. The deep-internals question Mistral and Cohere loops use to find real practitioners.

The GPU-utilization math that decides this argument, the hidden line items (ops headcount, quality gap, model churn), and the hybrid recommendation that wins both the analysis and the customer.

The experience probe that opens half of all applied-AI loops, and the place resumes go to die. The four-beat walkthrough structure interviewers can actually score, plus the two omissions that quietly downgrade you from architect to API caller.

Two compression levers with very different bills. The arithmetic that decides between them, when to stack both, and the eval discipline that keeps a 'cheaper' model from quietly tanking your hardest 10% of queries.

Million-token windows didn't kill RAG; they changed when you reach for it. The decision rule that holds up in production, and the silent failure each option hides behind a confident answer.

Three stages, one KL leash. The reward-model-plus-PPO loop, the beta*KL term that keeps the policy from drifting off the reference, and how DPO collapses it into a single classification loss by treating the reward as beta*log(pi/pi_ref). The objectives, written correctly.

Budget covers maybe 5M of 100M LLM calls. Run a cheap first stage on everything, route only low-confidence items to the LLM, and use those LLM labels to distill a student that absorbs most of the volume. The cost-vs-coverage math that decides where the threshold goes.

The warm-up that quietly filters out candidates who only know LLMs as a magic API. Nail the two-phase inference picture and you set up every later answer about latency, cost, and hallucinations.

Everyone can define it; few candidates know why 'advertised context' and 'usable context' aren't the same thing. The cost, latency, and attention caveats that make this answer stand out.

Modern LLMs share one architectural skeleton. Learn the five components and you can read almost any model card cold, plus the single distinction that proves to an interviewer you actually understand it rather than the buzzwords.

JSON mode, strict schemas, constrained decoding, validate-and-retry: the escalation ladder for structured output, plus the catch nobody mentions: schema-valid can still be factually wrong.

Every FDE will face an angry customer asking this. The mechanistic explanation interviewers want (why hallucination is the default behavior, not a bug) plus the framing that defuses customer escalations.

TTFT vs tokens-per-second, the output-length lever everyone forgets, and why streaming is the highest-ROI fix that changes no latency at all. The five-lever answer with real numbers.