Robots Atlas>ROBOTS ATLAS
Safety

Refusal

2022ActivePublished: 18 November 2026Updated: 18 November 2026Published
Key innovation
Turning the decision to decline a request into a learned, steerable model behaviour instilled by safety training (RLHF, safety fine-tuning) rather than a hard-coded rule.
Category
Safety
Abstraction level
Pojฤ™cie
Operation level
InferencePost-training
Use cases
Enforcing safety and content policiesBlocking illegal or harmful requestsContent moderation in AI assistantsLimiting misuseSafety evaluation and red-teaming

How it works

Refusal is instilled during post-training: human preferences teach the model to reject requests judged harmful (RLHF โ€” Bai et al., 2022), complemented by safety fine-tuning and principle-based methods (Constitutional AI). At inference time the model recognises a request as policy-violating and generates a refusal response instead of substantive content. Mechanistic analysis (Arditi et al., 2024) across 13 open-weight models of up to 72B parameters showed that this behaviour is represented by a single direction in the residual stream (the refusal direction): ablating the direction removes refusals, while adding it induces refusals even for harmless prompts. Adversarial suffixes used in jailbreaks suppress this direction, which the authors read as evidence that current safety fine-tuning is fragile.

Problem solved

Lets a model decline harmful, dangerous, or policy-violating requests without a rigid rule-based input filter, limiting misuse and potential harm while preserving the model's general usefulness.

Components

Safety training signalInstils the refusal behaviour during post-training (RLHF, safety fine-tuning, Constitutional AI).

Human preferences and safety data teach the model to reject requests judged harmful, rather than encoding an explicit rule.

Refusal directionMechanistic representation of refusal โ€” a single residual-stream direction that controls the behaviour.

Per Arditi et al. (2024), ablating the direction disables refusals while amplifying it induces refusals even on harmless prompts.

Evolution

2022
Helpful and harmless assistant trained with RLHF
Inflection point

RLHF trains the assistant to refuse harmful requests โ€” the basis of refusal as a learned behaviour (Bai et al., Anthropic).

2023
XSTest: identifying exaggerated safety (over-refusal)
Inflection point

A suite of 250 safe and 200 unsafe prompts exposing systematic over-refusal (NAACL 2024).

2024
Refusal mediated by a single direction
Inflection point

Arditi et al. show a refusal direction in the residual stream and a white-box jailbreak that disables refusals.

2024
OR-Bench: large-scale over-refusal benchmark

The first large-scale over-refusal benchmark (~80k prompts), measured across 32 models from 8 families (ICML 2025).