05·Early stopping and training monitoring strategies

Introduction

Early stopping is the simplest and one of the most effective regularization techniques: we stop training when validation metric stops improving, saving the best-epoch model. Despite simplicity, the mechanism has pitfalls: how long to wait for improvement (patience), which metric to focus on (loss vs accuracy vs F1), whether to reset weights at plateau, how to combine early stopping with a learning rate scheduler. The lesson walks through the full training-monitoring workflow in PyTorch (ModelCheckpoint, EarlyStopping in Lightning), classic plots (train/val loss curves), interpretation of different curve shapes (overfitting, underfitting, double descent Nakkiran et al. 2019), and integration with other regularization mechanisms (dropout, weight decay, BN). We also cover checkpoint averaging (SWA, EMA), techniques popular in production LLMs (Smith et al. 2022 — training even past overfitting until train_loss plateau), and rules of thumb for when to disable early stopping in generative tasks.