Values are mapped from higher to lower precision using a scale (and zero point), often per-channel or per-group. Quantization can be post-training (PTQ, e.g. GPTQ, AWQ) or quantization-aware (QAT); modern accelerators support FP8/FP4 formats in hardware.
Large models require huge memory and bandwidth; full precision is costly and hinders deployment on resource-constrained devices.
