Evaluation

LabBench

2026ActivePublished: 16 October 2026Updated: 16 October 2026Published

Key innovation

A benchmark that shifts evaluation from interpreting lab data to deciding the next step — whether an AI agent can reason from experimental evidence to the right next experiment.

How it works

The benchmark consists of 20 equally weighted tasks. In each task the agent is given laboratory evidence and must identify the right next experiment. The answer is graded against a rubric of criteria; the task score is the fraction of criteria passed. The headline aggregate metric is the "mean task score" — the average across all 20 task scores. Only complete, fully reviewed 20-task configurations are reported, and invalid outputs count as failures.

Problem solved

Frontier AI agents can interpret laboratory data but rarely decide well which experiment to run next. Existing scientific benchmarks mostly measure knowledge and interpretation rather than the ability to plan the next experimental step. LabBench targets this gap by evaluating the reasoning that leads to choosing the next experiment.

Components

20-task experimental-reasoning suiteBenchmark evaluation set

Twenty equally weighted tasks in which the agent reasons from laboratory evidence to the right next experiment.

Rubric grading and mean task score metricScoring and aggregation methodology

Each task is graded against a rubric of criteria; the task score is the fraction of criteria passed, and the aggregate metric is the mean across the 20 tasks.