04·Polynomial regression and hyperparameters

Introduction

When the y-x relationship is not linear, you can still use linear regression — after feature transformation. Polynomial regression expands X with x^2, x^3, ..., x^d (and interactions x_i·x_j) and fits a linear model in the new space. In this lesson we cover: feature engineering (PolynomialFeatures in sklearn), choosing degree d as a hyperparameter, why high d leads to overfitting (the model memorizes noise), Ridge/Lasso/ElasticNet regularization as defenses against overfit, train/val/test split + k-fold cross-validation, GridSearchCV. We show the classic U-shaped curve of test_error vs. d (the Bias-Variance trade-off in practice).