Qwen2-72B-Instruct

Qwen2-72B-Instruct is an instruction-tuned large language model from the Qwen2 family, developed by the Qwen team at Alibaba. It is built on the Qwen2-72B base model and post-trained with supervised fine-tuning (SFT) and Direct Preference Optimization (DPO). The weights are released publicly under the Tongyi Qianwen license (not Apache 2.0).

The architecture is a Transformer decoder with SwiGLU activation, QKV bias, and Grouped Query Attention (GQA), applied across all Qwen2 sizes for higher speed and lower memory usage. The model supports a context window of up to 131,072 tokens via YARN (length extrapolation for inputs beyond 32,768 tokens). It supports English, Chinese, and 27+ additional languages (including Polish, German, French, Spanish, Russian, Japanese, Korean, Arabic).

On benchmarks it scores 82.3 on MMLU, 64.4 on MMLU-Pro, 86.0 on HumanEval, 91.1 on GSM8K and 59.7 on MATH, outperforming Llama-3-70B-Instruct on most knowledge, coding and math tasks. Running it in BF16/FP16 requires multiple GPUs (~145 GB VRAM) or quantization; it runs locally and in the cloud via vLLM, Ollama or llama.cpp.