Transformer from Scratch · Self-Attention from Scratch
Query, Key, Value
Self-Attention from Scratch
Introduction
Query, Key and Value are three projections of the same input representation. In this lesson we separate their roles: Query asks, Key describes what can be matched, and Value carries the information retrieved into the output.