02·Tensor operations: addition, multiplication, matrix multiplication

Introduction

You can do three kinds of things with tensors: combine elementwise (addition, Hadamard multiplication), reduce along axes (sum, mean), and multiply matrices (matmul) — the operation that builds entire neural networks. In this lesson we separate elementwise multiplication (star *) from matrix multiplication (at-sign @), show the shape rule (3,4)·(4,5)=(3,5), introduce broadcasting as "automatic growth" of the smaller tensor to match the larger, and explain the intuition of a matrix as a transformation of a vector. These are all the operations you will see in the forward pass of every network from R3 onward.