A tensor is split into blocks of 32 consecutive elements. Each block gets one shared scaling factor in E8M0 format (an 8-bit power-of-two exponent), typically derived from the block's maximum absolute value. Elements are then scaled and stored in 8-bit FP8: E4M3 (4 exponent bits, 3 mantissa bits, range ~±448) is typically used for the forward pass, while E5M2 (5 exponent bits, 2 mantissa bits, wider range) is used for gradients in the backward pass. Matrix multiplications run on Tensor Cores that natively multiply FP8 values while accounting for the block scales, with accumulation in higher precision. The scale-conversion (rounding) algorithm materially affects how closely results match BF16.
FP8 quantization with a single per-tensor factor loses accuracy when a tensor spans a wide range of magnitudes (e.g. outliers in activations or gradients). MXFP8 addresses this by giving each 32-element block its own scaling factor, allowing models to be trained and served in 8 bits without quality loss and without modifying the training recipe.
A group of 32 consecutive tensor values that share a single scaling factor.
8-bit exponent-only value representing the per-block scaling factor as a power of two.
The 8-bit values: E4M3 for higher precision (forward), E5M2 for wider range (gradients).
How the E8M0 factor is computed and rounded materially affects accuracy; a poor algorithm diverges from BF16.
Full native acceleration is available only on Blackwell Tensor Cores; older GPUs (Hopper, Ada) require emulation.
32-element blocks are directional, so transposing a tensor requires recomputing scales (often two representations).
The OCP consortium (AMD, Arm, Intel, Meta, Microsoft, NVIDIA, Qualcomm) publishes the MX standard defining MXFP8; the originating paper shows the first generative LLM training with sub-8-bit weights, activations and gradients.
NVIDIA Blackwell introduces hardware support for microscaling (MXFP8, MXFP6, MXFP4) directly in the Tensor Cores, unlike emulation on earlier GPUs.
NVIDIA publishes recipes showing that MXFP8-E4M3 with a suitable conversion algorithm matches BF16 training for models up to 8B parameters on datasets up to 15T tokens.
Space complexity: ~8 bitów/element + 8 bitów na 32 elementy (E8M0).
Number of elements sharing one scaling factor. Fixed at 32 in the MX standard.
FP8 variant used for elements: E4M3 or E5M2.
Format of the shared scaling factor; in MX this is E8M0 (powers of two).
MXFP8 is a numeric format used in dense matrix multiplications on Tensor Cores; scaling operates at block level and accumulation happens in higher precision.
Block scaling and MXFP8 matmuls are fully parallel on GPU Tensor Cores, in both training and inference.
NVIDIA Blackwell Tensor Cores natively support MXFP8 matmuls with block scaling.
