Inference
Wanda (pruning)
2023ActiveUpdated: 22 September 2026Published
Key
innovation
Using activation norm alongside weight magnitude as a pruning criterion without retraining.
Category
Inference
Abstraction level
Building block
Operation level
Post-trainingInference
Use cases
LLM compressionWeight sparsityDeployment on constrained hardware
How it works
For each weight a score |weight| × input-activation norm is computed; per output, the lowest-scoring weights are removed, with no retraining.
Problem solved
Classic magnitude-only pruning works poorly for LLMs, while second-order methods are expensive.