02·Bias-variance tradeoff — the formal anatomy of error

Introduction

The bias-variance decomposition splits the expected prediction error into three components: squared bias (systematic model error), variance (sensitivity to the specific training set realization), and irreducible noise. This lesson lays out the formal decomposition for MSE, its interpretation, how different model classes sit in the bias-variance space, how dataset size and capacity affect each component, and why bagging reduces variance while boosting reduces bias. We build on classic works: Geman, Bienenstock & Doursat 1992 ("Neural networks and the bias/variance dilemma"), Hastie, Tibshirani & Friedman "ESL" 2009 (chapter 7).