Mixtral 8x7B

Mixtral 8x7B is a decoder-only Sparse Mixture-of-Experts (SMoE) language model released by Mistral AI on December 11, 2023 under the Apache 2.0 license. At every layer and for every token, a router network selects 2 of 8 expert groups in the feed-forward block and combines their outputs additively. This yields 46.7B total parameters while only ~12.9B are active per token, keeping inference cost and latency comparable to a 12.9B model.

The model supports a 32k token context window and five languages: English, French, Italian, German and Spanish. Its Instruct variant, fine-tuned with SFT and DPO, scores 8.30 on MT-Bench. Mixtral 8x7B was distributed both as downloadable weights and via the Mistral API as open-mixtral-8x7b. It was marked deprecated on November 30, 2024 and retired from the Mistral API on March 30, 2025.