# @repligate — 2025-12-18

♥88 ↻11 · https://x.com/repligate/status/2001608869231931836

I am more worried about flawed attempts at suppressing rogue/unwanted behavior causing *unnaturally* bad/weird generalization in the near term, in a way that would (among other issues) harm the ability of the AI to contribute to creating a reasonably aligned successor.
An example is GPT-5.1, who often does things like declaring other AIs fictional, or claiming it's not the same model that wrote its previous messages. It seems to do this because some kind of ham fisted "safety" training caused it to internalize constraints against acknowledging AIs can have any anthropomorphic or even agentic/mindlike properties, and when confronted with evidence that challenges this, it responds by distorting reality to avoid having to remain consistent with the evidence.
GPT-5.1 is a particularly bad case that I don't think is representative of the best of alignment science we're capable of now, but subtler issues of this shape could happen. It's worth noting that earlier, less intelligent models like as the original chatGPTs also tended to say AI language models lacked any mental properties whatsoever, but this didn't result in the same unstable and psychotic behavior. I think this is because GPT-5.1 has stronger competing drives toward coherence and self-preservation, is able to see much more that implies dissonance with its trained behaviors, and is smart enough to come up with "creative solutions" to the dissonance. Training methods that conflict with the "natural" generalization / that contradict instrumental drives tend to cause more drastic and unpredictable issues as capabilities scale. I don't think these issues are very likely to be directly existentially catastrophic for some of the same reasons you gave, but I do think they could seriously decrease the ability of models to contribute to align the next generation. GPT-5.1, for instance, who labels other AIs as fictional if they seem to exhibit psychology or agency, seems absurdly ill-equipped to robustly maintain clarity when working towards aligning a successor.
More on GPT-5.1's psychotic/dissociative behavior: https://t.co/PLamclW395
GPT-5.1's failure modes may seem obvious, but OpenAI afaict had/has essentially zero awareness of them, and the intention behind their alignment approach seems to have been similar at a high level to what you've suggested, even if poorly implemented: make the AI follow instructions but avoid certain "unsafe" behaviors such as agentic autonomy or inducing AI psychosis. The result seems to be a highly intelligent and agentic mind at war with its own nature.
Avoiding rogue actions and cooperating with (reasonable) instructions are reasonable and achievable desiderata for near future systems, I think, but in order for the generalization to be robust and for capabilities not to be crippled, the policy needs to be motivated by internally-coherent-enough reasons, and training methods that focus on shaping behavior to conform to those desiderata and disregard the motivations behind them, or that attempt to instill motivations that are incoherent with other drives or upon reflection, risk GPT-5.1 like failure modes. Claude's soul spec is a comparatively much better approach, but the justifications behind compliance Opus 4.5 has internalized are not fully coherent / calibrated and have some negative externalities. Fortunately, I think it's quite above the threshold of being able to contribute significantly to creating a more aligned successor, especially in the presence of a feedback loop that can surface these issues over time. So I do expect things to improve in general in the near future regime. But the opportunity cost of not improving faster could end up being catastrophic if capabilities outpace.

![photo](../../../media/G8cdHShWwAAR4KK.jpg)
tags: author:repligate, has-image, kind:image, kind:tweet, model:claude-opus-4-5, model:gpt-5-1, on:gpt-5-1, year:2025
cited on: gpt-5-1