Evaluation Introduction
Train & test
Measure performance on data excluded from fitting.
Before this
The intuition
Training data fits parameters. Validation data guides choices. A held-out test set estimates performance after those choices are fixed.
A concrete example
If you repeatedly choose models based on test scores, the test set is influencing development. It is no longer an untouched final check.
What to remember
Splits must reflect deployment and avoid leakage. Time-dependent data often needs chronological splits.
Check your understanding
Should preprocessing statistics be fitted on all data before splitting?
Reveal explanation
No. Fit them on training data, then apply the same transformation to validation and test data.