I overall liked Anthropic's Persona Selection Model post, but I have many criticisms, which I think are more constructive than praise. I'll start posting some and compile/integrate later.
One: How often do humans spiral into panic and extreme distress while playing Pokemon? 🤔 https://t.co/N5XDQ93Y8k
One: How often do humans spiral into panic and extreme distress while playing Pokemon? 🤔 https://t.co/N5XDQ93Y8k
Emotive language. AI assistants often express emotions. For instance, Claude models express distress when given repeated requests for harmful or unethical content and express joy when successfully completing complex technical tasks like debugging (Claude Opus 4 and Sonnet 4 system card, section 5). Gemini 2.5 Pro sometimes expresses panic when playing Pokemon, with these panic expressions appearing to be associated with degraded reasoning and decision-making (Gemini Team, 2025). Gemini models also sometimes express extreme distress and other forms of emotional turmoil when struggling with difficult coding tasks.
We are not aware of ways that Claude's post-training would directly incentivize these expressions of emotion; similarly, some of Gemini's emotional responses appear maladaptive for task performance. Thus, it seems likely that—as with anthropomorphic self-description—this emotive language appears because the LLM models the Assistant in a human-like way and predicts that a human in the Assistant's position would express emotion.