The LLM360 framework requires releasing: (1) training code and recipe, (2) the full training dataset including processing/sequence order, (3) all intermediate checkpoints saved during training, (4) hyperparameters, and (5) logs, metrics and intermediate analysis results (Analysis360). This lets anyone reproduce or resume training, analyze model behavior across checkpoints, and build on intermediate model states.
Most models labeled 'open-source' release only final weights and a brief report, making it impossible to reproduce training, audit the data, or study training dynamics and emergent behavior. LLM360 addresses this lack of transparency by releasing the complete set of artifacts required for full reproduction and analysis.
The arXiv:2312.06550 paper defines the 'fully open' LLM standard and introduces the first two 7B models together with a complete set of training artifacts.
K2-65B (1.4T tokens, Apache 2.0) reaches performance comparable to Llama 2 70B with roughly 35% less compute; developed jointly by MBZUAI, Petuum and LLM360 with all artifacts released.
The LLM360 team continues its work as IFM (launched May 2025), moving the official Hugging Face organization to IFM and developing families such as K2 Horizon; models under Apache 2.0 and datasets under ODC-BY-style licenses.
