02·Sequences, Tokens and Representations

Introduction

Before implementing attention, you need to understand sequences, tokens, vocabularies, embeddings and masks. These concepts determine tensor shapes in PyTorch.