Robots Atlas>ROBOTS ATLAS
Training

LLM360

2023ActivePublished: 4 September 2026Updated: 4 September 2026Published
Key innovation
Established a 'fully open' LLM standard: releasing not only the weights but the entire training process — data, code, recipe, intermediate checkpoints and training logs/metrics — for full reproducibility.
Category
Training
Abstraction level
Paradigm
Operation level
TrainingData
Use cases
Reproducible large language model researchAnalysis of training dynamics and intermediate checkpointsAuditing and inspection of training dataResuming or continuing training from checkpointsDevelopment of fully open foundation models

How it works

The LLM360 framework requires releasing: (1) training code and recipe, (2) the full training dataset including processing/sequence order, (3) all intermediate checkpoints saved during training, (4) hyperparameters, and (5) logs, metrics and intermediate analysis results (Analysis360). This lets anyone reproduce or resume training, analyze model behavior across checkpoints, and build on intermediate model states.

Problem solved

Most models labeled 'open-source' release only final weights and a brief report, making it impossible to reproduce training, audit the data, or study training dynamics and emergent behavior. LLM360 addresses this lack of transparency by releasing the complete set of artifacts required for full reproduction and analysis.

Components

Training code and recipeDefines how the model is trained and allows the process to be reproduced from scratch.

The complete pre-training code together with the recipe: architecture, hyperparameters and run configuration, enabling faithful reproduction of the training run.

Full training dataset (with data order)Provides transparency and auditability of the model’s source of knowledge.

The complete training data together with the order/sequence in which it is fed to the model, enabling data auditing and exact reproduction of the same learning trajectory.

Intermediate model checkpointsEnables analysis of the training trajectory and building on intermediate states.

All intermediate weight states saved during training (not only the final checkpoint), allowing study of training dynamics, emergent phenomena, and resuming training from any point.

Training metrics and logsDocuments the training run for verification and comparison.

Recorded metrics, learning curves and logs from the training run, documenting the behavior of the model and the optimization process over time.

Analysis360 (analysis suite)Provides ready-made tooling and analysis results over the training artifacts.

A companion project/repository providing analyses of intermediate checkpoints and results, supporting research into model behavior across training stages.

Official

Evolution

Original paper · 2023 · arXiv 2023 · Zhengzhong Liu
LLM360: Towards Fully Transparent Open-Source LLMs
Zhengzhong Liu, Aurick Qiao, Willie Neiswanger, Hongyi Wang, Bowen Tan, Tianhua Tao, Preslav Nakov, Tim Baldwin, Eric P. Xing
2023
Release of LLM360 and the Amber (7B) and CrystalCoder (7B) models
Inflection point

The arXiv:2312.06550 paper defines the 'fully open' LLM standard and introduces the first two 7B models together with a complete set of training artifacts.

LLM360: Towards Fully Transparent Open-Source LLMs (paper)
2024
Release of K2 (65B) — a fully reproducible 65B model

K2-65B (1.4T tokens, Apache 2.0) reaches performance comparable to Llama 2 70B with roughly 35% less compute; developed jointly by MBZUAI, Petuum and LLM360 with all artifacts released.

2025
Continuation under the Institute of Foundation Models (IFM)

The LLM360 team continues its work as IFM (launched May 2025), moving the official Hugging Face organization to IFM and developing families such as K2 Horizon; models under Apache 2.0 and datasets under ODC-BY-style licenses.