An independent AI evaluation company from San Francisco building industry-specific benchmarks that test whether models can do real professional work — in law, finance, medicine and engineering.
Vals AI describes itself as the independent evaluator of artificial intelligence. The company builds benchmarks and evaluation infrastructure to test whether models can perform professional-level work in fields such as law, banking, engineering and medicine. Its starting point is a criticism of the state of the field: in Vals AI's view, model benchmarks are seriously lacking, because they measure abstract intelligence rather than the industry-specific tasks on which models are actually used.
Rather than relying on third-party sets, the company develops and administers its own evaluations. It publishes composite indices — the Vals Index, the Vals Multimodal Index and the RSI Index covering recursive self-improvement capability — alongside domain benchmarks. In law these include Legal Research Bench, Case Law (v2) and runs of Harvey's Legal Agent Benchmark; in finance Finance Agent v2, CorpFin v2 and Mortgage Tax; in tax Tax Agent Bench and TaxEval v2; in technical work SWE-bench Verified, Terminal-Bench, Vibe Code Bench and CyberBench; in medicine MedCode and MedScribe. Public Benefits Bench, which measures the reliability of answers about SNAP assistance, occupies a place of its own.
The methodology rests on running models against identical task sets with standardized grading rubrics. The company states that it uses no fallback models on most benchmarks and counts refusals as failures. Beyond accuracy it measures cost per test, latency, token usage and tool-use capability, and analyses errors qualitatively. Test sets remain private so they do not reach training data, while validation datasets are released. Vals AI also open-sources its own infrastructure: Valkyrie, a distributed agentic benchmarking system, and a model library providing standardized API access. The company is headquartered in San Francisco, employs between 11 and 50 people, remains privately held and is backed by Andreessen Horowitz.
Classification
