Lesson 2 of 3 · About 5 minutes · Data leakage

Refit preprocessing inside every fold

Cross-validation gives different rows the validation role in each run. A scaler fitted once on all the rows has already seen every validation fold.

Try it

Switch the validation fold. Compare the mean fitted on all rows with the mean fitted inside the current training folds. Which rows may influence each fit?

Refit inside each fold

One fold is held out for validation in this run. Predict which values should set the mean, then change the fitting rule and rotate the held-out fold.

Validation fold

Fit preprocessing

Fitted mean: 11.00 · 10 rows

Fold 5 is marked for validation, but its values helped fit this mean. Fitting once before splitting crosses the boundary.

Data boundary for this validation run
Fold	Values	Role	Fits mean?
1	2, 4	Training	Yes
2	6, 8	Training	Yes
3	10, 12	Training	Yes
4	14, 16	Training	Yes
5	18, 20	Validation	Yes

Reusing rows in different runs is expected. Within each run, fit learned preprocessing on its training folds. These means show the fitting boundary, not a change in validation accuracy.

For each run, start with a fresh pipeline. Fit the preprocessing on that run's training folds, transform both training and validation rows with those fitted parameters, then fit the model on training rows and score it on validation rows. Validation scores may select a pipeline; the final test set stays outside that selection. Reusing observations across separate cross-validation runs is expected.