03·MQA, GQA and Lower Inference Cost

Introduction

You will learn Multi-Query Attention and Grouped-Query Attention: techniques that reduce K/V cache cost during generation without fully giving up multiple query heads.