02·MSE cost function

Introduction

Mean Squared Error (MSE) is the default regression loss. In this lesson we decompose it: why squared (and not absolute), what the cost surface looks like (a convex paraboloid) and why convexity is invaluable for optimization, what units it has (squared units of y), how it relates to RMSE and MAE, when it fails (outliers, heavy tails), and how it leads to the bias-variance decomposition. We show its equivalence with MLE under Gaussian noise and the geometric intuition: an orthogonal projection of y onto the column space of X.