03·Scaled Dot-Product Attention

Introduction

Scaled dot-product attention is the concrete formula used in the Transformer: QK^T scores are scaled, masked, normalized with softmax and multiplied by V. This lesson breaks down every step.