Lost in the Middle
How it works
The phenomenon is studied by systematically moving the position of relevant information within a long input while keeping context length fixed. In multi-document QA the model receives a question and many passages, only one of which contains the answer; accuracy is measured as a function of that passage’s position. Similarly, in key–value retrieval the model must read the value for a given key from a long list. The results form a U-shaped curve: high accuracy for edge positions, a minimum for middle positions. The causes are linked to attention distribution and positional encoding (primacy/recency bias) in the Transformer architecture, and to the distribution of context length and information placement in training data.
Problem solved
It refutes the tacit assumption that a model “sees” the whole context window uniformly. It names and characterizes a previously under-examined limitation of long-context LLMs: merely making information available in the context does not guarantee it will be used, because the position of that information strongly affects the outcome. This provides a basis for designing RAG and prompting systems that deliberately place key content at the edges of the context.
Implementation
In RAG systems, relevant passages that land in the middle of a long context are often ignored, lowering answer quality.
Enlarging the context window does not remove the U-curve; the model may still fail to use middle information.
A larger number of competing passages intensifies the effect — relevant information gets “lost” in the noise.
Evolution
The paper describes the U-shaped performance curve as a function of the relevant information’s position in a long context, on multi-document QA and key–value retrieval tasks.
The peer-reviewed version appeared in the Transactions of the Association for Computational Linguistics, cementing “Lost in the Middle” as a standard term in long-context research.
The “Lost in the Middle” findings inspired benchmarks measuring the real usable length of long context (e.g. needle-in-a-haystack tests and RULER).
Hyperparameters (configurable axes)
Where the relevant information sits within the context (beginning, middle, end). The key factor producing the U-shaped curve.
Input length and the number of competing documents. The longer the context, the stronger the drop for middle positions.
The model’s nominal context window and architecture. The effect appears even in large-context models (extending the window does not remove it).
Computational complexity
Compute bottleneck
The bottleneck is not compute throughput but the model’s ability to actually focus attention on a relevant passage located far from the edges of the input. As context length and the number of competing passages grow, middle-positioned information is used increasingly poorly.
Execution paradigm
The phenomenon is observed in dense self-attention models, where all input positions are processed but not used equally.