QSA restricts attention computation to relevant token connections and operates at the micro-block level in a hybrid architecture (alongside Gated DeltaNet and a 'Gated Residual' mechanism), lowering the cost of serving contexts of hundreds of thousands to a million tokens.
Dense attention is too costly for very long contexts; an efficient sparse-attention mechanism that preserves long-context quality is needed.
