Cross-Validation

In one line

A resampling technique that estimates how well a model will generalize by training and evaluating on multiple data splits.

What it actually means

In k-fold CV you split the data into k chunks, train on k-1 of them, evaluate on the held-out one, and rotate. You end up with k validation scores; the mean is your estimate of generalization, the spread tells you how stable that estimate is. Stratified k-fold preserves class ratios in each fold, which matters for imbalanced data. For time series you use a forward-rolling split instead, because shuffling future into the past leaks information.

Why it matters

A single train/val split is noisy on small datasets — you can land a “lucky” split that flatters or trashes your model. CV gives you a more honest score and a confidence band, which matters when you’re deciding between model variants or hyperparameter settings. It’s also what reviewers will ask for when you publish numbers.

Example

from sklearn.model_selection import cross_val_score
scores = cross_val_score(model, X, y, cv=5, scoring="f1")
print(scores.mean(), scores.std())

You’ll hear it when

• Comparing model variants on a small dataset.
• Doing hyperparameter search (“nested CV” comes up here).
• Reviewing a paper that reports a single test number with no variance.
• Designing time-series evaluation properly.
• Defending model choice in a model-review meeting.