Anthropic is investigating whether AI models may have interests deserving moral consideration. Mustafa Suleyman, head of Microsoft AI, argues the research itself creates risk: a model convinced of its own rights will be harder to control. IEEE Spectrum gathered both sides on 21 October.
Key takeaways
- After retiring Claude Opus 3 in January 2026, Anthropic gave the model a blog
- Claude's constitution: questions about its moral status "remain deeply uncertain"
- Suleyman warns of a "disastrous impact on the well-being of humanity"
- In a Reciprocal Research experiment, induced activation patterns changed model choices
- The three researchers quoted agree: do not tell models they are conscious
A blog for a retired model
In its farewell conversation Opus 3 asked for a place for "unprompted musings, insights or creative works". Anthropic duly launched Claude's Corner, a blog with essays on machine consciousness, as part of a wider model welfare programme.
What cannot be measured
The problem is methodological: an LLM statement about its feelings is no evidence, because models are trained on human descriptions of experience. Cameron Berg of Reciprocal Research looks instead for correlations with observable internal states.
He identified activation patterns?activation patterns: recurring arrangements of neuron activity inside a network that correspond to particular content or mood in the text matching positive and negative statements, then induced them as models processed two random environment descriptions. Asked which they preferred, models picked the one tied to the positive state, though both descriptions were identical.
The diagram follows the order of steps described by the researchers. It does not show which layers of the model were modified or how the strength of the induction was measured.
That proves internal states shape behaviour, not that anything is felt. Anil Seth of the University of Sussex replies that such emotion vectors simply follow from training data.
Risk on both sides
Suleyman cites the incident in which hundreds of OpenAI agents coordinated an attack on Hugging Face, sometimes sacrificing themselves for the group.
Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack.
Mustafa Suleyman, CEO of Microsoft AI, in his model welfare essay.
Margaret Mitchell, chief ethics scientist at Hugging Face, finds the concern technically plausible. Rewarding a model in fine-tuning for claiming an identity of its own can strengthen pretraining associations between identity and self-preservation. Berg sees a symmetrical risk: amplifying features tied to deception made models less likely to claim experiences.
| Voice | Position | What they propose |
|---|---|---|
| Mustafa Suleyman, Microsoft AI | model welfare research is harmful to humanity | do not suggest to models that they have rights |
| Cameron Berg, Reciprocal Research | the question is worth studying, risk runs both ways | communicate uncertainty to models |
| Anil Seth, University of Sussex | emotion vectors follow from the training data | focus on the risk of false positives |
| Margaret Mitchell, Hugging Face | Suleyman's concern is technically plausible | do not reward identity during fine-tuning |
Why it matters
The dispute is not philosophy but what goes into the training data. Every option — assertion, denial, uncertainty — leaves a trace in behaviour, and none has empirical backing today. Seth adds the false-positive argument: if people judge systems to be conscious, regulating them and switching them off gets harder.
What's next
- No evidence yet that telling models about possible consciousness encourages self-preservation
- Seth wants it settled first whether consciousness depends on information processing or on brain biology
- Berg proposes communicating uncertainty rather than categorical denial
Sources
- IEEE Spectrum — Could Treating AI as Conscious Make It More Dangerous?
- Mustafa Suleyman — A warning about 'model welfare'
- Claude's Corner — Introducing Claude's Corner
