Robots Atlas>ROBOTS ATLAS
Evaluation

APEX-Agents

2026ActivePublished: 18 November 2026Updated: 18 November 2026Published
Key innovation
Shifts AI productivity measurement from single expert questions (APEX) to long-horizon, multi-step agentic tasks performed in realistic environments with files and tools.
Category
Evaluation
Abstraction level
Benchmark
Operation level
Evaluation (runtime)Agent runtimeApplication
Use cases
Evaluating agentic capabilities of AI models and agentsComparing agents on long-horizon professional tasksMeasuring progress in knowledge-work automationBenchmarking agents for banking, consulting and legal tasks

How it works

Each task is embedded in an environment containing input files and available tools, and the agent must perform multi-step work until it delivers an output. The agent's output is graded against rubrics and gold outputs, and performance is reported with Pass@1 (the share of tasks solved on a single attempt). Agent execution and grading are handled by the open-source Archipelago infrastructure. Tasks are drawn from three professional domains: investment banking, consulting and corporate law.

Problem solved

Prior professional benchmarks (including APEX) evaluated models on single questions and did not test whether an agent can carry a realistic, multi-step task to completion using files and tools. APEX-Agents fills this gap by measuring agent productivity on tasks that resemble actual professional work.

Evolution

Original paper · 2026 · arXiv 2601.14242 (Mercor) · Bertie Vidgen
APEX-Agents
Bertie Vidgen
2025
APEX (The AI Productivity Index) released

Mercor releases APEX, a benchmark evaluating frontier models on single expert-level questions from four professions: investment banking, consulting, corporate law and medicine.

2026
APEX-Agents released
Inflection point

Mercor extends APEX to agentic tasks: 480 long-horizon, multi-step tasks in realistic environments with files and tools; the top Pass@1 score is 24.0%.