02·Decision trees — splits, Gini, and pruning

Introduction

A decision tree is a non-linear classifier that recursively partitions feature space with axis-aligned splits. Each split is a question "x_i ≤ t?", and leaves contain class predictions. This lesson dissects the CART algorithm (Breiman et al. 1984): Gini impurity, entropy, information gain, stopping criteria, post-pruning, and why trees are insensitive to feature scaling but extremely prone to overfitting without regularization.