Robots Atlas>ROBOTS ATLAS
Inference

EAGLE-3

2025ActivePublished: 19 November 2026Updated: 19 November 2026Published
Key innovation
Removes the feature-prediction constraint of EAGLE/EAGLE-2 and replaces it with direct token prediction plus multi-layer feature fusion, letting the draft model fully benefit from scaling up training data.
Category
Inference
Abstraction level
Technika
Operation level
InferenceServing
Use cases
LLM inference accelerationLow-latency LLM servingIncreasing generation throughput in production systemsDecoding for chat and reasoning models

How it works

1) The draft model generates a tree of candidate tokens. 2) Instead of predicting target-layer features (as in EAGLE-1/2), EAGLE-3 directly predicts tokens, using fused features from the target model's low, mid, and high layers. 3) The "training-time test" technique simulates the draft model's multi-step generation during training, reducing the train-inference mismatch. 4) The target model verifies the entire candidate tree in a single pass; the longest correct paths are accepted, guaranteeing output identical to standard decoding (lossless).

Problem solved

Autoregressive decoding in LLMs is memory-bound: each token requires a separate model pass, so generation underuses GPU compute. Earlier EAGLE versions sped this up, but their gains scaled poorly with training data due to feature-prediction constraints. EAGLE-3 addresses this scalability problem, letting the draft model improve as training data grows.

Components

Draft modelProposes candidate tokens verified by the target model

A lightweight autoregressive network producing cheap token proposals that the target model verifies in a single pass.

Multi-layer feature fusionFeeds the draft model a richer input signal

Combines representations from the target model's low, mid, and high layers instead of relying solely on top-layer features.

Training-time testDraft model training procedure

Simulates the draft model's multi-step generation during training, reducing the train-inference mismatch and enabling the removal of feature prediction.

Draft tree and verificationVerification of many candidates in a single pass

Candidate tokens are arranged into a tree (dynamic draft trees from EAGLE-2) and verified in parallel; the longest correct paths are accepted.

Implementation

Implementation pitfalls
Draft model train-inference mismatchMedium

A draft model trained without simulating multi-step generation achieves a lower acceptance rate at inference.

Fix:Use the training-time test technique, which is the core of EAGLE-3.
Dependence on alignment with the target modelMedium

The draft model is trained per target model; acceptance rate and speedup depend on distribution alignment.

Fix:Train the draft model on data/distribution matching the target model.

Evolution

Original paper · 2025 · NeurIPS 2025 · Yuhui Li
EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
Yuhui Li, Fangyun Wei, Chao Zhang, Hongyang Zhang
2023
EAGLE-1 released
Inflection point

Introduced feature-level autoregression for speculative sampling (ICML 2024).

EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty (paper)
2024
EAGLE-2 released

Dynamic draft trees improving the acceptance rate (EMNLP 2024).

EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees (paper)
2025
EAGLE-3 released
Inflection point

Dropped feature prediction in favour of direct token prediction and multi-layer feature fusion; training-time test technique (arXiv March 2025, accepted to NeurIPS 2025).

Hyperparameters (configurable axes)

Draft tree size / tokens per stepHigh

Number of candidate tokens in the tree verified per target-model pass; affects acceptance rate and speedup.

Fused feature layersMedium

Choice of target-model layers (low/mid/high) fused as the draft model's input.

Parallelism

Parallelism level
Partially parallel

Many candidate tokens (the draft tree) are verified in parallel in a single target-model pass, while the draft model itself remains autoregressive.

Scope
InferenceAcross tokens

Hardware requirements

Primary

Mitigates the memory-bound nature of decoding on GPUs by making better use of compute; benchmarked on hardware such as RTX 3090 GPUs.

Good fit

The method operates at the decoding-algorithm level and requires no specific hardware; it has been integrated into many serving frameworks.