Mixtral 8x22B

Mixtral 8x22B is an open-source Sparse Mixture-of-Experts (SMoE) language model developed by Mistral AI and released on 17 April 2024. It has 141 billion total parameters, of which only 39 billion are activated per token, delivering much faster inference than dense models of comparable quality.

The model supports a 64K-token context window and is natively multilingual — fluent in English, French, Italian, German and Spanish. It stands out for strong mathematics and coding abilities, as well as native function calling support.

Mixtral 8x22B was released under the Apache 2.0 license — one of the most permissive open-source licenses — with downloadable weights and access via la Plateforme (Mistral AI’s console). It can therefore be run both locally and in the cloud.

In benchmarks, the instructed version achieves 90.8% on GSM8K (8-shot) and 44.6% on MATH (maj@4). The model outperforms LLaMA 2 70B on multilingual benchmarks (French, German, Spanish, Italian) and achieves top results on coding tasks (HumanEval, MBPP).