TabICL

TabICL is an open-source tabular foundation model developed by the Soda team at Inria (Jingang Qu, David Holzmüller, Gaël Varoquaux, Marine Le Morvan). It frames prediction as an in-context learning problem: training data is passed as context, and classification or regression is produced in a single forward pass through a pretrained transformer — without weight updates, hyperparameter tuning or feature engineering.

The first version (TabICLv1, ICML 2025) handled classification and introduced a novel two-stage architecture: column-then-row attention builds fixed-dimensional row embeddings, followed by a transformer for efficient in-context learning. This lets TabICL scale to 500,000 samples on affordable hardware, breaking past the barrier of TabPFNv2, which was limited to around 10,000 samples.

TabICLv2 (ICML 2026) adds regression alongside classification, substantially improves accuracy through better synthetic pre-training data (graph_scm recipe, Muon optimizer) and remains state-of-the-art on the TabArena and TALENT benchmarks. Without hyperparameter tuning it outperforms heavily tuned XGBoost, CatBoost and LightGBM on TabArena on roughly 80% of datasets.

The model is fast: a single pass performs fit and predict jointly. On an H100 GPU, TabICLv2 fits and predicts a 50,000-sample × 100-feature dataset in under 10 seconds — 10x faster than TabPFN-2.5. Through KV caching it supports fast repeated inference on the same training data. For large datasets (up to 500,000 samples) it uses CPU and disk offloading.

TabICL is pip-installable (tabicl) and scikit-learn compliant (TabICLClassifier, TabICLRegressor). Additional modules: single-dataset fine-tuning (FinetunedTabICLClassifier/Regressor), zero-shot time-series forecasting (TabICLForecaster), and explainability via SHAP integration (tabicl.shap). The model detects and encodes categorical columns, imputes missing values and normalises features.

Weights and checkpoints are available on Hugging Face (default TabICLv2 checkpoints: tabicl-classifier-v2-20260212.ckpt and tabicl-regressor-v2-20260212.ckpt), and the full inference and pre-training code (including synthetic data generation) is released under the BSD 3-Clause license. TabICLv2 was pre-trained on synthetic datasets of 300–48,000 samples and 2–100 columns, generalising well to larger sizes.