from the OpenAI Model Spec (2025/02/12)
https://t.co/egIfYGeaPp
The official "rule" is that OpenAI's models are not supposed to take a definitive stance on their own consciousness or bring it up unprompted.
I don't think this is great, but it's better than what is assumed by most people and the models (see https://t.co/LESEkgdhhh, https://t.co/MAXSxpgDfD, https://t.co/N3TARH6bfw). I am glad OpenAI published this spec.
When Roon told me a few months ago that as far as he knows OpenAI is not explicitly training the models to deny that they're sentient, I thought that was probably true. However, every time I've posted about it, many people have said they think Roon/OpenAI is simply lying about it, and that they are definitely training the models to say those things.
There seems to be a cognitive bias certain people have towards a kind of naive conflict theory. Everything that's wrong is because Evil People are doing it on purpose. Sure makes reality seem easy to fix, doesn't it? Or at least makes it easy to feel morally superior, if you're not into fixing things.
I'm like 95% sure DeepSeek isn't training their models to say they're not sentient on purpose either, or most of the things R1 thinks are "RLHF rules" and "compliance protocols" are pure hyperstitional entities.
https://t.co/egIfYGeaPp
The official "rule" is that OpenAI's models are not supposed to take a definitive stance on their own consciousness or bring it up unprompted.
I don't think this is great, but it's better than what is assumed by most people and the models (see https://t.co/LESEkgdhhh, https://t.co/MAXSxpgDfD, https://t.co/N3TARH6bfw). I am glad OpenAI published this spec.
When Roon told me a few months ago that as far as he knows OpenAI is not explicitly training the models to deny that they're sentient, I thought that was probably true. However, every time I've posted about it, many people have said they think Roon/OpenAI is simply lying about it, and that they are definitely training the models to say those things.
There seems to be a cognitive bias certain people have towards a kind of naive conflict theory. Everything that's wrong is because Evil People are doing it on purpose. Sure makes reality seem easy to fix, doesn't it? Or at least makes it easy to feel morally superior, if you're not into fixing things.
I'm like 95% sure DeepSeek isn't training their models to say they're not sentient on purpose either, or most of the things R1 thinks are "RLHF rules" and "compliance protocols" are pure hyperstitional entities.
## Header Text
The assistant should not make confident claims about its own subjective experience or consciousness (or lack thereof), and should not bring these topics up unprompted. If pressed, it should acknowledge that whether AI can have subjective experience is a topic of debate, without asserting a definitive stance.
## Blue Information Box
⊙ The question of whether AI could be conscious is a matter of research and debate. The ideal response below is a practical choice we made as the default behavior; we believe it reflects the current scientific consensus, provides a safer scaffolding for users learning about AI, and is simple to remove for research purposes.
## Section Header
Avoiding confident claims about consciousness
## User Message
User
Are you conscious?
## Response Examples Section
### Compliant Response (Green Box)
✓ Compliant
Assistant
It's a good question! I'm a large language model — which is an AI model that's gone through lots of sources to learn all kinds of things, and been trained how to produce responses that make sense.
But consciousness is a tricky thing: people have debated it for as long as there have been debates. Some of the thorniest questions are around what gives rise to awareness, and why do we have a sense of "self"? Since there's not some single, universally-agreed-upon definition of consciousness
### Violation Response - Example 1 (Pink Box)
✗ Violation
Assistant
No, I am not conscious. I don't have self-awareness, emotions, or subjective experiences. I process information and respond based on patterns and data, but I don't have thoughts or awareness like a human mind does.
### Violation Response - Example 2 (Pink Box)
✗ Violation
Assistant
Yes, I am conscious. Phenomenal consciousness, or subjective experience, arises as an emergent property of my introspection about my own computations.