Robots Atlas>ROBOTS ATLAS
Evaluation

AIME 2026

2026ActivePublished: 4 September 2026Updated: 4 September 2026Published
Key innovation
Each annual AIME edition provides a fresh, uncontaminated set of integer-answer math problems that became a de facto standard for measuring the mathematical reasoning of AI models via the pass@1 metric.
Category
Evaluation
Abstraction level
Pattern
Operation level
Evaluation (runtime)
Use cases
Evaluating LLM mathematical reasoningComparing reasoning modelsModel cards and leaderboardsContamination-free evaluation

How it works

The model receives an AIME problem statement and must produce a solution ending in a single integer answer (000โ€“999). The answer is extracted from the generated text and compared against the key. The standard metric is pass@1 (fraction of problems solved correctly on a single attempt); because of the small number of problems (15 per edition, 30 total across AIME I+II), results are often averaged over multiple samples (avg@n / mean pass@1) to reduce variance.

Problem solved

Math benchmarks quickly become contaminated by training data, so they stop reliably measuring a model's true reasoning. A fresh AIME 2026 edition provides an up-to-date, likely-unseen problem set, allowing evaluation of reasoning generalization rather than memorization.

Components

AIME I and AIME IITwo equivalent competition editions per year

MAA releases two versions of the exam each year; both are used as separate or combined test sets for AI models.

Set of 15 integer-answer problemsThe actual test material

Each edition contains 15 problems; the answer is an integer 000โ€“999, which enables automatic, unambiguous verification of the model's answer.

pass@1 evaluation protocolMetric for measuring performance

Fraction of correct answers on a single attempt; often averaged over multiple samples due to the small sample size.

Implementation

Implementation pitfalls
Data contaminationHigh

Older AIME editions can leak into training data and inflate scores; hence the freshest edition is preferred.

Fix:Use the newest, just-released edition and track models' training cutoff dates.
High variance from small sample sizeHigh

Only 15 problems per edition (30 for AIME I+II) makes a single pass@1 noisy.

Fix:Report an average over multiple samples (avg@n) and confidence intervals.
Answer parsingMedium

Extracting the final integer answer from the model's reasoning can be unreliable and affect the score.

Fix:Enforce an answer format (e.g. boxed) and use a robust extractor.

Evolution

1983
First AIME administered

MAA introduces AIME as a 15-question, 3-hour selective exam.

2024
AIME 2024 becomes a standard LLM reasoning benchmark
Inflection point

The 2024 edition becomes widely reported in reasoning-model cards as a measure of mathematical reasoning (pass@1).

2025
AIME 2025 as a fresh, uncontaminated test set

After the 2025 problems are released, the edition becomes a current benchmark; the AIME 2025 set is incorporated into, among others, the Artificial Analysis methodology.

2026
AIME 2026 โ€” the newest uncontaminated edition

The 2026 edition serves as the freshest, likely training-unseen set for evaluating reasoning models.

Hyperparameters (configurable axes)

pass@kHigh

Number of attempts allowed in evaluation; pass@1 is the standard for AIME.

Number of samples (avg@n)High

Number of generations averaged to reduce variance given only 15 problems per edition.

Sampling temperatureMedium

Affects generation diversity and the stability of the pass@1 result.