TL;DR: K-fold rotates the test role to get a stabler estimate on small data, and you report mean and spread. Refuse plain random k-fold on time-ordered data (use forward-chaining) and on grouped data (use GroupKFold), because shuffling either one leaks the future or the entity and inflates the score you hand the customer.
How to approach it
Frame CV as an answer to a specific problem: "With small datasets, a single train/test split gives a noisy performance estimate that depends on which rows landed in the test set. Cross-validation buys a stabler estimate by rotating the test role." Then volunteer the limits before being asked, that's where the signal is.
A strong answer
Mechanics in three sentences: split the data into k folds (k=5 or 10 typically); train on k−1, validate on the held-out fold; rotate so every fold validates once; report the mean and the standard deviation across folds. The spread matters as much as the mean, "0.82 ± 0.01" and "0.82 ± 0.09" are very different stories to tell a customer about pilot risk. Cost: k× training time, which is why on large datasets a single well-constructed holdout is usually fine, CV earns its keep below roughly 50k rows, exactly the regime most customer pilots start in.
When to refuse standard k-fold:
- Time-ordered data (forecasting, churn, fraud, most enterprise problems). Random folds train on the future to predict the past, inflating scores. Use time-series CV (expanding or rolling window): train on months 1–6, validate on 7; train on 1–7, validate on 8; and so on. Add a purge gap between train and validation windows if features use lagging aggregates.
- Grouped data. Multiple rows per customer/patient/device scattered across folds means the model memorizes entities. Use GroupKFold, all of an entity's rows stay in one fold. A patient-readmission model can lose 10+ points of AUC moving from naive k-fold to grouped CV; the grouped number is the honest one.
Forward-chaining is easiest to hold as the schedule it produces, so write it out once for a 10-month dataset with monthly folds and a 1-month purge gap (the gap matters when features use 30-day rolling aggregates, which would otherwise straddle the boundary):
| Fold | Train on | Purged | Validate on |
|---|---|---|---|
| 1 | months 1-5 | month 6 | month 7 |
| 2 | months 1-6 | month 7 | month 8 |
| 3 | months 1-7 | month 8 | month 9 |
| 4 | months 1-8 | month 9 | month 10 |
Two properties to say out loud: every validation month is strictly later than everything it was trained on, which is the deployment situation exactly; and the later folds train on more data, so a score that improves across folds is also telling you the model is data-hungry, a free learning-curve reading that random k-fold cannot give you. The price is fewer folds and a noisier estimate than shuffled CV, and that price is correct to pay, because the alternative is a smaller error bar around a number that is wrong.
Also mention stratified k-fold as the default for imbalanced classification (keeps the positive rate consistent per fold), and the cardinal rule: every preprocessing step that learns from data, scaling, encoding, imputation, feature selection, resampling, must be fit inside each fold, or you've laundered leakage into your estimate.
What interviewers probe next
- "How do you use CV for hyperparameter tuning and still get an honest final number?", nested CV, or more pragmatically: tune with CV on the training portion, then evaluate once on an untouched, later-in-time test set.
- "What does high variance across folds tell you?", the dataset is small or heterogeneous (e.g., one fold dominated by a single big customer segment); pilot results will be unstable and you should say so in the readout.
- "Leave-one-out?", k=n; nearly unbiased but expensive and high-variance; rarely worth it outside tiny scientific datasets.
- "Does CV replace a test set?", no. CV guides development; a final held-out, time-separated test set settles what you report to the customer.
Common mistakes
Describing the mechanics flawlessly and never mentioning time or groups, the actual point of the question for deployment-focused roles like Databricks and Google Cloud AI. Fitting the scaler once on all data "for convenience." Reporting only the fold mean and hiding the variance. And using CV scores directly in a customer readout after dozens of tuning iterations against them, that estimate is gently overfit, and the production gap will out you.