The Institute of Foundation Models released K2 Horizon on 3 September — six models from 0.9B to 375B parameters under Apache 2.0. The news is not the weights but what sits beside them: checkpoints, training data or its build recipes, code, configs and logs. The weights are already downloadable.
Key takeaways
- Six models: 375B-A23B, 36B-A4B, 32B, 7B, 3.7B and 0.9B — all under Apache 2.0
- The whole training lifecycle is open: checkpoints, data or recipes, code, configs, logs, eval results and final weights
- 36B-A4B uses MoVA, an in-house sparse attention mechanism, activating about 4B parameters per token
- The 0.9B variant clears 48 on AIME 2026 and fits on watches and glasses
- Datasets carry separate licences, including ODC-BY, rather than Apache 2.0
A fleet, not a single model
The six share architecture, vocabulary, training methodology, interfaces and deployment tooling — apart from a smaller vocabulary in the 0.9B variant. The largest, 375B-A23B, is a sparse MoE: 375 billion parameters of capacity, roughly 23 billion activated per token.
| Model | Parameters | Active / token | Target environment |
|---|---|---|---|
| 375B-A23B | 375B | ~23B | data centre, enterprise deployments |
| 36B-A4B (MoVA) | 36B | ~4B | workstation, efficient serving |
| 32B (dense) | 32B | 32B | local workstation |
| 7B | 7B | 7B | phone, on-device |
| 3.7B | 3.7B | 3.7B | phone, on-device |
| 0.9B | 0.9B | 0.9B | watch, glasses (under quantisation) |
MoVA: sparse attention instead of sparse FFN layers
Technically, the most interesting is 36B-A4B. IFM combined its own MoVA — Mixture-of-Value-Attention — with Mixture of Experts layers, activating around 4 billion parameters per token. The result is close to the dense 32B with eight times fewer active parameters. Having both in one family is a rare chance to compare dense and sparse architectures under identical training conditions.
What exactly is open
The full-openness claim comes with a concrete list: training data or its construction methods and mixtures, training code, configurations, intermediate checkpoints?Intermediate checkpoint: A snapshot of model weights taken during training rather than after it. It lets you trace when specific capabilities appear — and resume training from that point., fine-grained logs, evaluation results and final weights. Models and code go out under Apache 2.0, datasets under their own licences — where redistribution is impossible, IFM documents how the data was built. It continues the principle set out in the 2023 LLM360 paper.
Why it matters
Open weights let you run a model but say nothing about how it was made. An open training lifecycle turns a checkpoint from black box into research material — you can trace when planning or tool use emerges and repeat the method on your own data. For teams without a from-scratch training budget, that is the difference between copying and understanding.
What's next
- Weights for all six are already on Hugging Face, with GGUF and FP8 variants published on 3 September
- IFM notes tasks demanding long exploration and repeated recovery, such as TerminalBench, remain out of reach for the smallest models
Sources
- IFM — Introducing K2 Horizon: Frontier Performance, Radically Open
- Hugging Face — IFM/K2-Horizon-375B-A23B
