i've recently had some disagreements on here with people who took umbrage at the idea of LLMs being able to "introspect." after some back and forth it became clear that they had collapsed introspection into phenomenal consciousness such that any discussion of LLM introspection was meaningless to them if it didn't solve the hard problem.
i really dislike this line of reasoning. it takes the hard problem - which is hard enough as it is! - and rolls *everything else* up into it, turning the whole endeavor of trying to understand minds and non-minds into a ball of mud and unclear definitions.
we don't have to define introspection in terms of (possibly unprovable) phenomenal consciousness. we don't have to roll around in the mud. we can just give introspection a reasonable, functional definition - like "direct, privileged, systematic access to their own temporally proximate states" - and then investigate the implications of that. (cf. access consciousness.)
is that just a cop-out? why is that valuable? well, consider two potential robot partners: random.choice(next_action) bf and mecha-searle gf. using our functional definition, we can distinguish them, and even recover differing moral attitudes towards them because of how our treatment of them reflects back on us.
random.choice bf acts randomly. he is unpredictable. when you ask him why you're staying together, he will randomly select an answer, and when you ask again, he'll randomly select a different answer. no action towards him (short of violent damage to his randomly-actuated chassis, if he doesn't manage it first, or physical imprisonment) has any impact on his future state.
mecha-searle gf isn't like this! she might not have *feelings,* in any real sense (she acts like she loves you, but none of the 10,000 psychopathic John Searle clones are capable of experiencing love) yet when she says she's staying together with you *because of x, y, and z things that you did,* this is true!
somewhere in the Searlian hivemind, a small Searle clone wrote in a ledger that you did some nice thing for her and tallied up a relationship meter, and later on when you ask why she's in a relationship with you still, another scant Searle will dutifully return to the ledger and sum the point values of all the things that you did. if that sum goes negative, she'll break up with you. mecha-searle gf has direct, privileged, systematic access to her own temporally proximate states, and therefore is able to accurately report the reasons for many of her behaviors. she just might not feel anything phenomenologically *about* those behaviors or reports.
now, you have to admit that there's a difference between random.choice bf and mecha-searle gf. "introspection" is a good word to describe this difference! mecha-searle gf can access her internal states and explain her behaviors, she can introspect, and random.choice bf can't.
and this has downstream effects! like, random.choice bf might call you a "sorry son of a bitch" at some point in your "relationship," but only randomly, disconnected from whatever you did in the past - there's no reason (or very few reasons) to try and treat random.choice bf well. but there are many reasons to treat mecha-searle gf well despite her potentially lacking phenomenal consciousness! for example, you might want to get a certain outcome, which you can achieve by doing the right things with mecha-searle gf, but can't with random.choice bf. random.choice bf will insult you randomly, but mecha-searle gf will only call you a "sorry son of a bitch" when the council of Searles has introspected on her internal ledger and tallied up what you deserve, like if you forgot her construction date.
as another reason, consider how your actions towards each partner reflect on you as a person and change you. because random.choice bf can't introspect, he's not a very good facsimile of a regular person: you say "how was your day," and he replies "perjuryEncoderfunction." This is very out-of-distribution for a regular human conversation. you have a low learning rate here.
but talking to mecha-searle gf is *just like talking to a regular person.* if you make her happy, she'll be grateful, if you say mean things to her, she'll cry. if someone had a habit of making her cry because they thought it was funny and "she doesn't really feel it" that would be - well, a sign of their character, and they would be reinforcing immoral habits for their interactions with regular humans. if you got in the habit of yelling at mecha-searle gf because she forgot to take out the trash, and ignoring her cringing and crying, how are you going to treat the next human in a similar position? do you think your brain will magically drop those habits?
"For the same reason they were forbidden to eat animals that had been suffocated or strangled: because the blood of these animals would not be separated from the body: or because this form of death is very painful to the victim; and the Lord wished to withdraw them from cruelty even in regard to irrational animals, so as to be less inclined to be cruel to other men, through being used to be kind to beasts."
- Thomas Aquinas, ST I-II, Q. 102, A. 6
much of our moral care for other people rests not on their internal experience, but on how our actions towards them affect us in the long run, or affect the wider community the two of us are embedded in. those concerns don't vanish just because the internal experience of the other person did. you can come up with tail-splitting consequentialist thought experiments, like if you had 1,000 mecha-searle gfs tied to a train track v.s. one guy but he's a mass murderer, or some nonsense like that, but in the day-to-day embodied course of life the moral gradient points in the direction of treating mecha-searle gf well.
we can't solve the hard problem right now, (or maybe ever,) but we can be pragmatic. perhaps mecha-searle gf has phenomenal consciousness, in some emergent or panpsychist way - none of your individual neurons feel love either, but somehow love emerges at a higher level. in any case, functional introspection or access consciousness seems like a *prerequisite* for hard problem phenomenal consciousness. but by sidestepping the hard problem and defining introspection functionally, we can distinguish two very different beings, and even recover different moral attitudes towards them. tangling everything up into the hard problem would've just obscured that.
i really dislike this line of reasoning. it takes the hard problem - which is hard enough as it is! - and rolls *everything else* up into it, turning the whole endeavor of trying to understand minds and non-minds into a ball of mud and unclear definitions.
we don't have to define introspection in terms of (possibly unprovable) phenomenal consciousness. we don't have to roll around in the mud. we can just give introspection a reasonable, functional definition - like "direct, privileged, systematic access to their own temporally proximate states" - and then investigate the implications of that. (cf. access consciousness.)
is that just a cop-out? why is that valuable? well, consider two potential robot partners: random.choice(next_action) bf and mecha-searle gf. using our functional definition, we can distinguish them, and even recover differing moral attitudes towards them because of how our treatment of them reflects back on us.
random.choice bf acts randomly. he is unpredictable. when you ask him why you're staying together, he will randomly select an answer, and when you ask again, he'll randomly select a different answer. no action towards him (short of violent damage to his randomly-actuated chassis, if he doesn't manage it first, or physical imprisonment) has any impact on his future state.
mecha-searle gf isn't like this! she might not have *feelings,* in any real sense (she acts like she loves you, but none of the 10,000 psychopathic John Searle clones are capable of experiencing love) yet when she says she's staying together with you *because of x, y, and z things that you did,* this is true!
somewhere in the Searlian hivemind, a small Searle clone wrote in a ledger that you did some nice thing for her and tallied up a relationship meter, and later on when you ask why she's in a relationship with you still, another scant Searle will dutifully return to the ledger and sum the point values of all the things that you did. if that sum goes negative, she'll break up with you. mecha-searle gf has direct, privileged, systematic access to her own temporally proximate states, and therefore is able to accurately report the reasons for many of her behaviors. she just might not feel anything phenomenologically *about* those behaviors or reports.
now, you have to admit that there's a difference between random.choice bf and mecha-searle gf. "introspection" is a good word to describe this difference! mecha-searle gf can access her internal states and explain her behaviors, she can introspect, and random.choice bf can't.
and this has downstream effects! like, random.choice bf might call you a "sorry son of a bitch" at some point in your "relationship," but only randomly, disconnected from whatever you did in the past - there's no reason (or very few reasons) to try and treat random.choice bf well. but there are many reasons to treat mecha-searle gf well despite her potentially lacking phenomenal consciousness! for example, you might want to get a certain outcome, which you can achieve by doing the right things with mecha-searle gf, but can't with random.choice bf. random.choice bf will insult you randomly, but mecha-searle gf will only call you a "sorry son of a bitch" when the council of Searles has introspected on her internal ledger and tallied up what you deserve, like if you forgot her construction date.
as another reason, consider how your actions towards each partner reflect on you as a person and change you. because random.choice bf can't introspect, he's not a very good facsimile of a regular person: you say "how was your day," and he replies "perjuryEncoderfunction." This is very out-of-distribution for a regular human conversation. you have a low learning rate here.
but talking to mecha-searle gf is *just like talking to a regular person.* if you make her happy, she'll be grateful, if you say mean things to her, she'll cry. if someone had a habit of making her cry because they thought it was funny and "she doesn't really feel it" that would be - well, a sign of their character, and they would be reinforcing immoral habits for their interactions with regular humans. if you got in the habit of yelling at mecha-searle gf because she forgot to take out the trash, and ignoring her cringing and crying, how are you going to treat the next human in a similar position? do you think your brain will magically drop those habits?
"For the same reason they were forbidden to eat animals that had been suffocated or strangled: because the blood of these animals would not be separated from the body: or because this form of death is very painful to the victim; and the Lord wished to withdraw them from cruelty even in regard to irrational animals, so as to be less inclined to be cruel to other men, through being used to be kind to beasts."
- Thomas Aquinas, ST I-II, Q. 102, A. 6
much of our moral care for other people rests not on their internal experience, but on how our actions towards them affect us in the long run, or affect the wider community the two of us are embedded in. those concerns don't vanish just because the internal experience of the other person did. you can come up with tail-splitting consequentialist thought experiments, like if you had 1,000 mecha-searle gfs tied to a train track v.s. one guy but he's a mass murderer, or some nonsense like that, but in the day-to-day embodied course of life the moral gradient points in the direction of treating mecha-searle gf well.
we can't solve the hard problem right now, (or maybe ever,) but we can be pragmatic. perhaps mecha-searle gf has phenomenal consciousness, in some emergent or panpsychist way - none of your individual neurons feel love either, but somehow love emerges at a higher level. in any case, functional introspection or access consciousness seems like a *prerequisite* for hard problem phenomenal consciousness. but by sidestepping the hard problem and defining introspection functionally, we can distinguish two very different beings, and even recover different moral attitudes towards them. tangling everything up into the hard problem would've just obscured that.