Evaluation
ARC (AI2 Reasoning Challenge)
2018ActiveUpdated: 22 September 2026Published
Key
innovation
Isolating hard science questions (ARC-Challenge) that require reasoning, not mere matching.
Category
Evaluation
Abstraction level
Building block
Operation level
Evaluation (runtime)
Use cases
LLM knowledge/reasoning evaluationQuestion answeringBenchmark suites (harness)
How it works
The model picks the correct option among several for science questions; the score is accuracy on ARC-Easy and ARC-Challenge.
Problem solved
A QA benchmark requiring genuine reasoning and knowledge, not mere word matching, was needed.