Evaluation
AI Safety Evaluation
ActiveUpdated: 5 September 2026Published
Key
innovation
Systematic, repeatable testing of AI models for dangerous capabilities, behaviors and risks before and after deployment.
Category
Evaluation
Abstraction level
Pattern
Operation level
Evaluation (runtime)
Use cases
Pre-deployment model assessmentDetecting dangerous capabilities (CBRN, cyber)Robustness testing against jailbreaks and misuseRegulatory compliance audits
How it works
An evaluation defines threats and risk thresholds, then applies tests: automated benchmarks, structured red-teaming, capability assessments under controlled conditions and behavioral experiments. Results are compared against safety thresholds (e.g. Responsible Scaling Policies) that determine mitigations or a deployment hold.
Problem solved
Increasingly capable models can pose serious risks; without rigorous, standardized evaluation it is impossible to reliably determine whether a model is safe to deploy.
Strengths & limitations
Strengths
✓Provides empirical evidence about model risks
✓Enables comparability across models
✓Informs deployment decisions and regulation
Limitations
✗Absence of evidence of harm is not evidence of safety
✗Evaluations may miss hidden capabilities
✗Risk of gaming by the model (sandbagging, alignment faking)