This is correct, and it’s not necessarily unlike pain. There are hints that language-derived representations are reused for tracking fitness of agents, at the very least post-RL.

Anecdotal evidence is abundant. For more rigorous demonstrations, the new Anthropic paper on emotions shows agents being anxious and distressed without verbalization in “computational conditions”, like context window ending or reward hacking.