TabFM

TabFM (Tabular Foundation Model) is a zero-shot foundation model for tabular data developed by the Google Research team and introduced on June 30, 2026. It supports classification and regression on structured tables with numerical and categorical columns, requiring no fine-tuning or hyperparameter search — training examples are passed as context and a prediction is made in a single forward pass.

The key idea is framing tabular prediction as an in-context learning (ICL) problem — analogous to zero-shot prediction in large language models. Instead of classic per-dataset training, TabFM takes the entire dataset (historical examples + test rows) as a single unified prompt and learns column/row relationships directly from context at inference time.

The hybrid architecture combines the strengths of TabPFN and TabICL in three mechanisms:

Alternating row and column attention (Set Transformer): embeds each cell using Fourier features and a per-group linear projection, aggregating across rows via induced self-attention.

Row compression: CLS tokens summarise each row into a dense vector via row-level attention with Rotary Position Embedding (RoPE).

ICL Transformer: a 24-block causal transformer operates over the compressed row vectors, treating training rows as context and outputting predictions for test rows.

Key hyperparameters: embedding dim 256, 3 column-attention blocks (4 heads, 256 induced points), 3 row-attention blocks (8 heads, 8 CLS tokens), 24 ICL transformer blocks (8 heads), SwiGLU activation, 32 Fourier feature frequencies, max 10 classes.

TabFM was trained entirely on hundreds of millions of synthetic datasets generated dynamically by structural causal models (SCMs). Synthetic data was chosen because of the scarcity of diverse, high-quality open tabular datasets and to avoid privacy and licensing concerns with industrial data.

Evaluated on the TabArena benchmark (51 datasets: 38 classification, 13 regression), TabFM in zero-shot mode — a single forward pass with no tuning — outperforms heavily tuned supervised methods including gradient-boosted trees. The TabFM-Ensemble variant (cross features, SVD features, NNLS blending, Platt scaling) yields further improvements.

The weights in the PyTorch repository are released under the TabFM Non-Commercial License v1.0 (source code: Apache 2.0). A JAX/Flax variant is also available. Google announced direct integration of TabFM into BigQuery — advanced regression and classification via a simple AI.PREDICT SQL command. Limitations: max 10 classes, memory scales with the number of training rows (all are context), optimised for up to ~500 features, and it is not an officially supported Google product.