TL;DR: K-fold rotates the test role to get a stabler estimate on small data, and you report mean and spread. Refuse plain random k-fold on time-ordered data (use forward-chaining) and on grouped data (use GroupKFold), because shuffling either one leaks the future or the entity and inflates the score you hand the customer.
How to approach it
Frame CV as an answer to a specific problem: "With small datasets, a single train/test split gives a noisy performance estimate that depends on which rows landed in the test set. Cross-validation buys a stabler estimate by rotating the test role." Then volunteer the limits before being asked, that's where the signal is.
A strong answer
Mechanics in three sentences: split the data into k folds (k=5 or 10 typically); train on k−1, validate on the held-out fold; rotate so every fold validates once; report the mean and the standard deviation across folds. The spread matters as much as the mean, "0.82 ± 0.01" and "0.82 ± 0.09" are very different stories to tell a customer about pilot risk. Cost: k× training time, which is why on large datasets a single well-constructed holdout is usually fine, CV earns its keep below roughly 50k rows, exactly the regime most customer pilots start in.
When to refuse standard k-fold:
- Time-ordered data (forecasting, churn, fraud, most enterprise problems). Random folds train on the future to predict the past, inflating scores. Use time-series CV (expanding or rolling window): train on months 1–6, validate on 7; train on 1–7, validate on 8; and so on. Add a purge gap between train and validation windows when labels or features look forward past the boundary.
- Grouped data. Multiple rows per customer/patient/device scattered across folds means the model memorizes entities. Use GroupKFold, all of an entity's rows stay in one fold. The grouped number estimates performance on entities the model has never seen, which is the honest one when that is what deployment looks like (new patients, new customers); say which situation you are in, because the grouped score is not guaranteed to be lower.
Forward-chaining is easiest to hold as the schedule it produces, so write it out once for a 10-month dataset with monthly folds and a 1-month purge gap (the gap matters when a label looks forward, say churn within the next 30 days, because a training row's label would then be computed from the validation month; a backward-looking 30-day feature that exists at prediction time needs no gap):
| Fold | Train on | Purged | Validate on |
|---|---|---|---|
| 1 | months 1-5 | month 6 | month 7 |
| 2 | months 1-6 | month 7 | month 8 |
| 3 | months 1-7 | month 8 | month 9 |
| 4 | months 1-8 | month 9 | month 10 |
Two properties to say out loud: every validation month is strictly later than everything it was trained on, which is the deployment situation exactly; and the later folds train on more data and validate on a different month, so a score that improves across folds does not by itself say the model is data-hungry: drift, seasonality or a changing class mix can move it too (an unchanged always-negative classifier scores 60%, 70%, 80%, 90% if later months hold that share of negatives). To ask whether more data helps, run a learning curve: train on growing windows and score them all on the same later month. The price is fewer folds and a noisier estimate than shuffled CV, and that price is correct to pay, because the alternative is a smaller error bar around a number that is wrong.
Also mention stratified k-fold as the default for imbalanced classification (keeps the positive rate consistent per fold), and the cardinal rule: every preprocessing step that learns from data, scaling, encoding, imputation, feature selection, resampling, must be fit inside each fold, or you've laundered leakage into your estimate.
What interviewers probe next
- "How do you use CV for hyperparameter tuning and still get an honest final number?", nested CV, or more pragmatically: tune with CV on the training portion, then evaluate once on an untouched, later-in-time test set.
- "What does high variance across folds tell you?", the dataset is small or heterogeneous (e.g., one fold dominated by a single big customer segment); pilot results will be unstable and you should say so in the readout.
- "Leave-one-out?", k=n; nearly unbiased but expensive and high-variance; rarely worth it outside tiny scientific datasets.
- "Does CV replace a test set?", no. CV guides development; a final held-out, time-separated test set settles what you report to the customer.
Common mistakes
Describing the mechanics flawlessly and never mentioning time or groups, the actual point of the question for deployment-focused roles like Databricks and Google Cloud AI. Fitting the scaler once on all data "for convenience." Reporting only the fold mean and hiding the variance. And using CV scores directly in a customer readout after dozens of tuning iterations against them, that estimate is gently overfit, and the production gap will out you.