Kimi K3

Kimi K3 is Moonshot AI's flagship model released on 16 July 2026 — Moonshot positions it as the world's first open 3T-class model. A sparse Mixture-of-Experts with 2.8 trillion total parameters, activating 16 of 896 experts per token. Context window up to 1 million tokens, native multimodality covering text, image, and video. The model targets long-horizon coding, knowledge work, and reasoning. For nine of the past twelve months, Kimi models set the upper bound of open-model sizes.

The architectural core is Kimi Delta Attention (KDA) — a hybrid linear attention mechanism delivering up to 6.3x faster decoding in million-token contexts — and Attention Residuals (AttnRes), which works along the depth axis, selectively retrieving representations across layers rather than accumulating them uniformly. AttnRes yields roughly 25% higher training efficiency at under 2% additional cost. Sparsity is implemented by Stable LatentMoE (16/896 active). Key routing components: Quantile Balancing (expert allocation derived directly from router-score quantiles, eliminating heuristics and a sensitive balancing hyperparameter), Per-Head Muon (extends Muon by optimising each attention head independently), Sigmoid Tanh Unit (SiTU), and Gated MLA — improved activation control and attention selectivity.

Refined training and data recipes together yield roughly 2.5x better scaling efficiency than Kimi K2. Quantization-Aware Training (QAT) applied from the SFT stage onward. MXFP4 weights, MXFP8 activations for broad hardware compatibility. Recommended configuration is a supernode with 64 or more accelerators. Because KDA poses new challenges for prefix caching, Moonshot contributed an implementation to vLLM. The model also runs on SGLang and llama.cpp.

Benchmarks (reasoning_effort=max, harness KimiCode/Claude Code/Codex): DeepSWE 67.5%, Program Bench 77.8%, Terminal Bench 2.1 88.3% (second only to GPT-5.6 Sol at 88.8%), FrontierSWE 81.2%, SWE Marathon 42.0%, BrowseComp 91.2% (with context compaction @300K tokens), Automation Bench 30.8%, GPQA-Diamond 93.5%, HLE-Full 43.5%, MMMU-Pro 81.6%, OmniDocBench 91.1%. K3 leads on Program Bench, SWE Marathon, BrowseComp, Automation Bench, and OmniDocBench; it trails Claude Fable 5 on FrontierSWE and HLE-Full, and GPT-5.6 Sol on DeepSWE. Moonshot is direct that K3 still trails the most powerful proprietary models (Fable 5, GPT-5.6 Sol), but consistently outperforms other tested open models.

Availability: kimi.com, Kimi Work, Kimi Code, and API (base_url: api.moonshot.ai/v1, OpenAI SDK compatible). Four API rules: reasoning_effort supports only 'max' (the K2.x 'thinking' parameter must not be used), temperature/top_p/n are fixed (omit them), max_completion_tokens defaults to 131,072 and reaches 1,048,576, in multi-turn and tool calls return the complete assistant message. Flat pricing (no tiering by context length): cache-hit input $0.30/MTok, cache-miss input $3.00/MTok, output $15.00/MTok. Moonshot reports a cache-hit rate above 90% in coding workloads. Sample use cases: repo-scale engineering (long sessions, minimal oversight), vision-in-the-loop (iterating between code and live screenshots), research reproduction (I-Love-Q relations: 20+ papers, 3000+ lines of Python), deep research reports (42-year ASIC study: 2.8k+ fetches, 11k+ pages), document parsing (OmniDocBench 91.1). Moonshot AI is backed by Alibaba and Tencent.