Qwen3-14B

Qwen3-14B is a post-trained (instruct) language model from the Qwen3 family developed by the Qwen Team at Alibaba Group and released on 29 April 2025 under the Apache 2.0 licence. It is a dense model with 14.8 billion parameters (13.2B non-embedding), built on 40 Transformer layers with GQA (40 Q heads, 8 KV heads). The model is part of the Qwen3 series, which includes dense models from 0.6B to 32B and MoE models at 30B–235B.

Hybrid thinking mode

The key feature of Qwen3 is support for two modes within a single model: thinking mode (enable_thinking=True) — the model generates step-by-step reasoning inside a <think>…</think> block, followed by the final answer; and non-thinking mode (enable_thinking=False) — a direct answer without explicit reasoning. The mode can be switched dynamically via /think and /no_think flags in the prompt or the enable_thinking parameter in the chat template. Recommended settings for thinking mode: Temperature=0.6, TopP=0.95, TopK=20. For non-thinking mode: Temperature=0.7, TopP=0.8, TopK=20.

Architecture and pretraining

The architecture is based on Transformer with GQA, SwiGLU, RoPE and RMSNorm, with added QK-Norm for training stability and without QKV-bias. The model natively supports 32K-token context, extendable to 131,072 with YaRN (commonly presented as 128K). Pre-training covered approximately 36 trillion tokens in three stages: a general stage (~30T), a STEM/code-heavy stage (~5T) and a long-context stage.

Post-training

Qwen3 post-training comprised 4 stages: (1) Long-CoT cold start — fine-tuning on chain-of-thought data from mathematics, code and STEM; (2) Reasoning RL — reinforcement learning with rule-based rewards (GRPO); (3) Thinking Mode Fusion — integration of the non-thinking mode via SFT on mixed CoT + instruction data; (4) General RL — RL across 20+ general tasks (instruction following, format compliance, agent tasks). Smaller dense models were partly trained via strong-to-weak knowledge distillation from larger Qwen3 models.

Multilingual support and agent capabilities

The model supports 119 languages and dialects, covering Indo-European, Sino-Tibetan, Afro-Asiatic, Austronesian and other language families. For agentic use, Qwen3-14B integrates with external tools via MCP (Model Context Protocol), supporting function calling in both thinking and non-thinking modes. Qwen-Agent is the recommended agent framework.