Qwen3-8B

Qwen3-8B is a post-trained (instruct) language model from the Qwen3 family developed by the Qwen Team at Alibaba Group and released on 29 April 2025 under the Apache 2.0 licence. It is a dense model with 8.2 billion parameters (6.95B non-embedding), built on 36 Transformer layers with GQA (32 Q heads, 8 KV heads). The model is part of the Qwen3 series, which includes dense models from 0.6B to 32B and MoE models at 30B–235B.

Hybrid thinking mode

The key innovation of Qwen3 is support for two modes within a single model: thinking mode (enable_thinking=True) — the model generates step-by-step reasoning inside a <think>…</think> block, followed by the final answer; and non-thinking mode (enable_thinking=False) — a direct answer without explicit reasoning, similar to traditional chat models. The mode can be switched dynamically via /think and /no_think flags in the prompt or the enable_thinking parameter in the chat template. Recommended settings for thinking mode: Temperature=0.6, TopP=0.95, TopK=20. For non-thinking mode: Temperature=0.7, TopP=0.8, TopK=20.

Architecture and pretraining

The architecture is based on Transformer with GQA, SwiGLU, RoPE and RMSNorm (similar to Qwen2.5), but without QKV-bias and with added QK-Norm for training stability. The model natively supports 128K-token context (extendable to 131K with YaRN). Pre-training covered approximately 36 trillion tokens in three stages: a general stage (30T tokens), a STEM/code-heavy stage (5T tokens) and a long-context stage (up to 32K, extended to 128K). Extended synthetic data was generated using Qwen2.5-VL, Qwen2.5-Math and Qwen2.5-Coder models.

Post-training (4 stages)

Post-training of the model included 4 stages: (1) Long-CoT cold start — fine-tuning on CoT data from mathematics, code and STEM. (2) Reasoning RL — reinforcement learning with rule-based rewards (GRPO). (3) Thinking Mode Fusion — integration of the non-thinking mode via SFT on mixed CoT + instruction data. (4) General RL — RL across 20+ general tasks (instruction following, format compliance, agent tasks). Smaller models (including 8B) were trained via strong-to-weak distillation from larger models instead of the full 4-stage pipeline.

Multilingual support and agent capabilities

The model supports 119 languages and dialects (vs. 29 in Qwen2.5), covering Indo-European, Sino-Tibetan, Afro-Asiatic, Austronesian and other language families. In terms of agent capabilities, Qwen3-8B integrates well with external tools via MCP (Model Context Protocol), supporting function calls in both thinking and non-thinking modes. Qwen-Agent is the recommended agent framework.