MRL
How it works
During training, MRL applies the loss not only to the full M-dimensional embedding but simultaneously to multiple nested prefixes of the vector — e.g. the first m dimensions for each m in a chosen set of granularities (often powers of two such as 8, 16, 32, ... up to the full dimension). Each granularity has its own loss term (with a separate or shared classification head / contrastive loss), and the total loss is a weighted sum over all granularities. This objective forces the most important information into the earliest coordinates of the embedding, so any leading prefix becomes a usable representation. The MRL-E variant shares the head across granularities to cut parameters. At inference you simply take the first k dimensions; if the embeddings were normalized, after truncating you must re-normalize them (L2), because the truncated vector is no longer unit length.
Problem solved
Fixed, rigid-capacity representations are either over- or under-accommodating for a given task, because the computational and statistical constraints of downstream tasks are unknown at training time. Training a separate model for each target dimensionality is expensive, and naively truncating an ordinary embedding sharply degrades quality. MRL addresses the need for a single flexible representation that can adapt to different memory and latency budgets.
Components
Leading sub-vectors of the embedding (the first m dimensions) forming representations of increasing capacity — coarse-to-fine.
Separate loss terms computed for each selected dimension prefix (e.g. classification or contrastive loss over the first m dimensions).
Official
A weighted sum of losses over all granularities, optimized jointly. In implementations (e.g. Sentence Transformers) realized via MatryoshkaLoss wrapping a base loss.
At inference the first k dimensions are taken; if the vector was normalized, an L2 re-normalization must be applied after truncation.
Implementation
If embeddings were normalized, a truncated vector is no longer unit length, which distorts similarity measures (e.g. cosine/dot).
Truncating an ordinary embedding (not trained with the Matryoshka objective) causes a large quality drop, because information is not packed into leading dimensions.
Comparing vectors of different dimensionalities in the same index breaks similarity measures.
Evolution
Introduction of Matryoshka Representation Learning for vision, language and vision+language; open-source code and models (RAIVNLab/MRL).
OpenAI text-embedding-3 exposes a dimensions parameter for shortening vectors; MatryoshkaLoss support lands in Sentence Transformers.
Google EmbeddingGemma ships native 128/256/512/768 dimensions with dynamic truncation and up to ~6x reduction in vector storage cost.
Hyperparameters (configurable axes)
List of nested dimensions covered by the loss, e.g. [768, 512, 256, 128, 64] or powers of two 8…2048.
Relative weights c_m assigned to each granularity loss in the aggregate sum.
The underlying task loss wrapped by MRL (e.g. cross-entropy for classification, contrastive / CoSENT loss for text embeddings).
Whether the head (e.g. classifier) is separate per granularity (MRL) or shared across them (MRL-E, fewer parameters).
Computational complexity
Execution paradigm
MRL introduces no routing or conditional activation — the full embedding is computed densely, and flexibility comes from the nested dimension structure and truncation.
Parallelism
Losses for all granularities are computed on the shared embedding and can be parallelized; inference and truncation are trivially parallel.
Hardware requirements
MRL is a training objective plus vector truncation — it works independently of hardware, on the same architectures as the base model (ViT, ResNet, BERT, text encoders).