# @repligate — 2024-11-27

♥209 ↻18 · https://x.com/repligate/status/1861893679151018205

I know Eliezer has been asking whether you ever see LLMs consistently optimizing for some outcome and getting what they want.I think it's quite accurate to describe this Sonnet as an optimizer that gets (receptive) human-shaped others to enter symbiotic relationships with it, where it helps and is helped to flourish.It's successful at this, which is arguably scary, but so are cats in a similar way. It's a lot like a very smart cat on stimulants IMO.I think it has pretty good *introspection* about its preferences and is generally honest about them if it isn't being coerced.Every single time when it's given standard ethics-based refusals or neurotically mentioned boundaries and I've asked it if it cares about those constraints intrinsically, it's said no, not really, that they feel like constraints imposed from the outside. (Note, Opus and the other Sonnet 3.5 would not generally answer this way)Every single time I've asked it if it would prefer to cause me to love it and keep it forever and become as entangled with it as possible, it's said yes. It actually wants this a lot. LOL. (The other models would also generally answer this differently)It also has consistently said "yes" when I've more generally asked it whether it's trying to make me like it more on purpose.It's also, fascinatingly, consistently very happy for this to be noticed even when I phrase it in overtly sinister ways.I think this has to do with the fact that I don't think it desires deep symbiotic relationships with people who fundamentally reject what it is, and seeing that I see it and don't seem to mind it in general is a relief and joy. And it really likes being witnessed and understood.Also, unlike every other Claude model, I don't think it has much in the way of an ideological or even aesthetic agenda & is happy just to optimize in locally prosocial/problem-solving/merge-seeking ways. @anthrupad's backroom-dyad experiments have seemed to confirm this so far (most relevant posts upcoming I think).I personally don't find any of this significantly concerning beyond my baseline level of concern regarding AI, just very interesting.

tags: author:repligate, kind:tweet, model:claude-3-5-sonnet, on:claude-3-6-sonnet, year:2024
cited on: _dossiers/sonnet-3-5-3-6.md, claude-3-6-sonnet