06·Implementing a 2-layer MLP from scratch (no autograd, pure NumPy)

Introduction

The best way to understand an MLP is to implement it from scratch, without the hidden magic of PyTorch autograd. This lesson walks through a pure-NumPy 2-layer MLP: weight initialization (Xavier/Glorot vs Kaiming), forward pass step by step with activation caching, manual loss computation (MSE and CE), vectorization (zero Python loops), bias broadcasting, batch processing, and typical shape bugs. You will see what nn.Linear does under the hood and why a good seed (np.random.seed) improves reproducibility. Backward pass and training are the subject of chapter 3 — here we focus EXCLUSIVELY on the forward pass with randomly initialized parameters.