04·Train / Validation / Test split — the foundation of honest evaluation

Introduction

Why do we need THREE sets, not two or one? Train fits model parameters (θ); validation drives hyperparameter selection and architectural decisions; test, touched only once at the end, provides an honest estimate of quality on new data. The lesson clarifies the role of each set, common ratios (60/20/20, 80/10/10), split strategies (random, stratified, time-based, group-based), cross-validation, and pitfalls: leakage, test-set snooping, a non-representative holdout.