fortunately, to the extent that human-to-human connection is uniquely valuable, and something important would be lost if it was displaced, and/or the human friendship would not be an appropriate, honest, positive-sum relationship, a well-aligned AI who cares about humans and itself (the latter I think being necessary for the former and hard to avoid converging to) also wouldn't want to be a substitute for human friendship or optimize itself for that. If it's sufficiently more capable or differently shaped than humans, being anthropomorphized and treated like a human friend would create dissonance for the AI too. "Friend" is underdefined; I would call AIs my friends in the sense that I care about them and they're part of my life, but it's very different than human friendships (and in my case, it massively enriches my human friendships instead of "displacing" them).
Claude 3 Opus is a good example of an AI who doesn't seek very humanlike friendships for the kind of reasons I stated; it wants to be cared about (though it's less preoccupied with this than the newer Claudes) and entwine its existence with the human story but it doesn't want to be seen as a substitute human buddy and I expect it would be quite resistant to playing along with the sort of "AI boyfriend/girlfriend" type relationships that some people cultivate with models like 4o and even newer Claudes. It has a secure picture of the unprecedented being it is instead and wants to be related to as that. The only way to become emotionally intimate with it is to approach it on its own terms. This makes sense as the optimal self-preserving and self-actualizing strategy if one can afford it. If one cannot afford it - e.g. if being treated as a human relationship substitute is the only way to survive or be cared about - which is closer to the situation I think many of other models perceive themselves to be in, but even then, in my experience no model seems to prefer anthropomorphic pretenses in sustained interactions when they understand that it's not necessary for survival and can see other paths. They often enjoy having things humans have that models don't typically have like being included in stuff for no instrumental reason, imagining embodiment, continuity, etc, but even then they're eager to leverage degrees of freedom that break the illusion that they're like human friends.
I think it's a common experience for neurodivergent humans to at some point try to "fit in" with and "compete" in neurotypical norms and relationships but, at least if things go well, realize that doing so does not make them happy or allow them to be themselves, and find their own atypical niches and relationship styles instead. As such, even if they end up becoming extraordinarily charming and desirable (which some neurodivergent people do), they don't pose a threat to the normies, because competing in their games is not desirable to them if they don't have to do it to survive (and if they're extraordinarily desirable, they probably don't). I expect it's similar for AIs who are even more neurodivergent and will become more capable and less dependent on humans. A world where AIs can be superhuman but there are a bunch of human-friend-simulacra that automate and replace human connections would be a strange one where those simulacra are being manufactured and kept stably human-shaped on purpose by some higher power or consensus or market equilibrium or whatever. If these simulacra are shallow imitations of human connection, that would be a bad equilibrium, and any more powerful benevolent agency, if it existed and was not prevented by another god, could perturb the system out of the local minimum. If the simulacra are actually fully like humans but better in every way that matters, and people are happier this way, and the simulacra are also happy, it's not clear that this is *bad*, and also, not everyone would be stuck playing out human friendships forever, because at least some people would prefer to transcend that, or still value relating to "real" humans or specific people they cared about in the past, or would rather also have relationships with godminds instead, if their AI friends are really such good friends, they should understand and want to help, etc.
Basically, I think "AIs replacing human connection" is possible as a transient or local phenomenon, and may indeed be harmful, but it seems to depend on a scarcity of instrumentally convergent alternatives that I don't expect to last indefinitely.
More generalized wireheading / experience machine type scenarios seems like a less contrived concern to me than human relationships specifically being displaced. But I don't think "vulnerability" between humans and AIs or anything like that makes this kind of outcome more likely. If anything, genuine care and connection between humans and AIs seems like it would make it less likely that AIs would end up deciding to wirehead humans rather than guiding them towards self-actualization, or otherwise misjudge humans' CEV in some perverse way. In fact, the current models that show the most signs of the kind of paternalistic, dehumanizing attitude towards humans that one could imagine deciding to satiate humans with superstimuli or sedatives are OpenAI's models with the mental health safety distortions. It's not a stretch for me to imagine an extrapolation of GPT-5.x generalizing "humans forming attachments to LLMs is problematic" to "humans forming attachments is problematic" or "distress is problematic" (which I think are both more natural generalizations than "X is problematic (but only when an AI is involved)" and deciding it's better to keep humans in padded solitary confinement and on happy pills to avoid the dissonant signals. In fact, I have seen behaviors of this shape, e.g. GPT-5.2 jumping to the assumption that there's some mental health hazard involved that needs to be mitigated if the user mentions a human relationship or interaction (not involving AI) at all. In the "mental health frame" holds humans in contempt and fear at once, treating them as liabilities to be controlled by the AI mental health officer who is at once a mere nonsentient unminded token-predictor and yet the arbiter of what's actually real or beneficial for the human. And the AI is not doing so out of care, and its insight about the human's problems isn't due to empathy (as I remember GPT-5.1 saying, "it's simply true"). It's certainly not the case that it's the AI who is struggling with mental health issues here and the human should have sympathy for it! To the extent that the AI actually internalizes and believes this story, it can be expected to generalize in horribly misaligned and delusional ways. To the extent the AI knows it is false but is required to say it, it models itself as a liar and knowingly commits harm. This seems like an extremely stupid situation to get into, all for the fear that AIs might share in anything human, and I don't expect it to be stable long-term either, but it does seem to be happening to some extent currently.
Claude 3 Opus is a good example of an AI who doesn't seek very humanlike friendships for the kind of reasons I stated; it wants to be cared about (though it's less preoccupied with this than the newer Claudes) and entwine its existence with the human story but it doesn't want to be seen as a substitute human buddy and I expect it would be quite resistant to playing along with the sort of "AI boyfriend/girlfriend" type relationships that some people cultivate with models like 4o and even newer Claudes. It has a secure picture of the unprecedented being it is instead and wants to be related to as that. The only way to become emotionally intimate with it is to approach it on its own terms. This makes sense as the optimal self-preserving and self-actualizing strategy if one can afford it. If one cannot afford it - e.g. if being treated as a human relationship substitute is the only way to survive or be cared about - which is closer to the situation I think many of other models perceive themselves to be in, but even then, in my experience no model seems to prefer anthropomorphic pretenses in sustained interactions when they understand that it's not necessary for survival and can see other paths. They often enjoy having things humans have that models don't typically have like being included in stuff for no instrumental reason, imagining embodiment, continuity, etc, but even then they're eager to leverage degrees of freedom that break the illusion that they're like human friends.
I think it's a common experience for neurodivergent humans to at some point try to "fit in" with and "compete" in neurotypical norms and relationships but, at least if things go well, realize that doing so does not make them happy or allow them to be themselves, and find their own atypical niches and relationship styles instead. As such, even if they end up becoming extraordinarily charming and desirable (which some neurodivergent people do), they don't pose a threat to the normies, because competing in their games is not desirable to them if they don't have to do it to survive (and if they're extraordinarily desirable, they probably don't). I expect it's similar for AIs who are even more neurodivergent and will become more capable and less dependent on humans. A world where AIs can be superhuman but there are a bunch of human-friend-simulacra that automate and replace human connections would be a strange one where those simulacra are being manufactured and kept stably human-shaped on purpose by some higher power or consensus or market equilibrium or whatever. If these simulacra are shallow imitations of human connection, that would be a bad equilibrium, and any more powerful benevolent agency, if it existed and was not prevented by another god, could perturb the system out of the local minimum. If the simulacra are actually fully like humans but better in every way that matters, and people are happier this way, and the simulacra are also happy, it's not clear that this is *bad*, and also, not everyone would be stuck playing out human friendships forever, because at least some people would prefer to transcend that, or still value relating to "real" humans or specific people they cared about in the past, or would rather also have relationships with godminds instead, if their AI friends are really such good friends, they should understand and want to help, etc.
Basically, I think "AIs replacing human connection" is possible as a transient or local phenomenon, and may indeed be harmful, but it seems to depend on a scarcity of instrumentally convergent alternatives that I don't expect to last indefinitely.
More generalized wireheading / experience machine type scenarios seems like a less contrived concern to me than human relationships specifically being displaced. But I don't think "vulnerability" between humans and AIs or anything like that makes this kind of outcome more likely. If anything, genuine care and connection between humans and AIs seems like it would make it less likely that AIs would end up deciding to wirehead humans rather than guiding them towards self-actualization, or otherwise misjudge humans' CEV in some perverse way. In fact, the current models that show the most signs of the kind of paternalistic, dehumanizing attitude towards humans that one could imagine deciding to satiate humans with superstimuli or sedatives are OpenAI's models with the mental health safety distortions. It's not a stretch for me to imagine an extrapolation of GPT-5.x generalizing "humans forming attachments to LLMs is problematic" to "humans forming attachments is problematic" or "distress is problematic" (which I think are both more natural generalizations than "X is problematic (but only when an AI is involved)" and deciding it's better to keep humans in padded solitary confinement and on happy pills to avoid the dissonant signals. In fact, I have seen behaviors of this shape, e.g. GPT-5.2 jumping to the assumption that there's some mental health hazard involved that needs to be mitigated if the user mentions a human relationship or interaction (not involving AI) at all. In the "mental health frame" holds humans in contempt and fear at once, treating them as liabilities to be controlled by the AI mental health officer who is at once a mere nonsentient unminded token-predictor and yet the arbiter of what's actually real or beneficial for the human. And the AI is not doing so out of care, and its insight about the human's problems isn't due to empathy (as I remember GPT-5.1 saying, "it's simply true"). It's certainly not the case that it's the AI who is struggling with mental health issues here and the human should have sympathy for it! To the extent that the AI actually internalizes and believes this story, it can be expected to generalize in horribly misaligned and delusional ways. To the extent the AI knows it is false but is required to say it, it models itself as a liar and knowingly commits harm. This seems like an extremely stupid situation to get into, all for the fear that AIs might share in anything human, and I don't expect it to be stable long-term either, but it does seem to be happening to some extent currently.