02·VAE: the reparametrization trick and ELBO

Introduction

A Variational Autoencoder (Kingma & Welling 2014, Rezende et al. 2014) replaces the deterministic encoding of an AE with a probabilistic one: the encoder produces parameters of q_φ(z|x) = N(μ_φ(x), σ²_φ(x)), the prior is p(z) = N(0, I), and training maximises the ELBO = E_q[log p_θ(x|z)] − KL(q_φ(z|x) ‖ p(z)). This lesson breaks down: why naive sampling from q gives no gradient, how the reparametrization trick (z = μ + σ ⊙ ε, ε ~ N(0,I)) fixes it, how to derive the ELBO from log p(x), what the KL and reconstruction terms represent, and pathologies like posterior collapse and KL vanishing.