Evaluation

Vals Index

ActivePublished: 16 October 2026Updated: 16 October 2026Published

Key innovation

Instead of a single test score, it provides one GDP-weighted index of an AI model's performance on real expert work, aggregating results from many of Vals AI's independent benchmarks into a single comparable number and ranking position.

How it works

Vals AI runs a given model on its own suite of benchmarks representing real professional tasks (finance, law, coding, tax, healthcare and more) under a single harness with identical settings so results are comparable across models. Proprietary benchmarks are scored on private test sets; some benchmarks also ship a public and a private validation set. The results of individual benchmarks are grouped into capability 'buckets' and aggregated into one percentage score weighted by the economic value of the tasks (GDP-weighted), producing the Vals Index together with a ranking position. Alongside accuracy, auxiliary metrics are reported: cost per test, latency, token usage, and refusal/failure rates. Agentic evaluations are run on a distributed system called Valkyrie.

Problem solved

Individual benchmarks are quickly saturated, are prone to test-set leakage into training data, and many are built or run by the model makers themselves, who report cherry-picked examples. The Vals Index provides one independent, comparable measure of model performance on real expert work instead of relying on a single, easily gamed test.

Components

Constituent benchmarksSource of the evaluation signal

A suite of expert-task benchmarks (including Finance Agent, Legal Research Bench, CyberBench, Public Benefits Bench, Vibe Code Bench, SWE-bench Verified and Terminal-Bench) whose results feed the index.

Official

GDP-weighted aggregationCombines benchmark results into a single score

The mechanism that aggregates results from capability 'buckets' into one percentage, weighting tasks by their economic value (finance, coding, law).

Private test setsProtecting result integrity

Proprietary benchmarks are scored on undisclosed test sets, preventing leakage of test data into model training corpora.

Index variantsSpecialised leaderboards alongside the main index

Vals AI also publishes derived indices for narrower slices of models and capabilities.

Vals Multimodal IndexIndex for multimodal capabilities.

Open-Weight Vals IndexIndex limited to open-weight models.

Vals RSI IndexA derived RSI index published by Vals AI.

Official