MiMo-V2.6-Flash-RL

MiMo-V2.6-Flash-RL is the efficiency-balanced checkpoint of the MiMo-V2.6 series developed by the Xiaomi MiMo team. It is an omnimodal model with a sparse Mixture-of-Experts architecture totaling 309 billion parameters, of which 15 billion are activated per forward pass (256 routed experts, 8 active). The model accepts text, image, video and audio, generates text (including code) and supports a context window of up to 1 million tokens.

Architecture

The LLM backbone has 48 transformer layers (39 sliding-window attention and 9 global attention) with a hidden size of 4096, 64 attention heads and 4 key-value heads (GQA). It combines omnimodal encoders: a MiMo ViT vision encoder (681M parameters, 28 layers) and audio encoders (308M tokenizer + 127M patch encoder). Speculative decoding is handled by a 5-layer Multi-Token Prediction drafter. Weights are released in FP8 format.

How the Flash variant differs

Flash-RL is the smaller, efficiency-balanced variant of the series — alongside the flagship MiMo-V2.6-Pro-RL (1.02T / 42B active, 384 experts) and the distilled MiMo-V2.6-Distill-Qwen-9B. It has 309B total and 15B activated parameters (256 experts, 8 active) and 48 layers instead of Pro-RL's 70, lowering deployment requirements (reference: 8 GPUs tensor-parallel × 2 data-parallel in SGLang, or 4 GPUs tensor-parallel in vLLM). This comes at a few-percent drop across most benchmarks; on CyberGym it scores marginally higher (95.1 vs 94.0). The model is trained with the same scaled reinforcement learning method (GRPO) with on-policy distillation (MOPD2) and a self-improvement loop.