(tho personally i enjoy some controversial schizo takes, or unexpected possibly true things, sprinkled with my straightforwardly true statements :p but super well-written!)
This is an interesting analysis! I think at the moment I disagree with it, though. Three points...
First, we do use theory-of-mind to understand fictional characters, but also constantly recognize that they are fictional. Chatbots aren’t fictional, and are interactive, so the way people relate to them is likely to be only partly analogous.
Second, there’s a descriptive vs normative issue. Although people naturally *do* use theory-of-mind to relate to chatbots, that doesn’t mean it is a good idea. The consequences of relating to them that way may be quite different from the consequences of relating to fictional characters that way.
Finally, whereas what you suggest may be true for “phenomenological study of the model's behaviour,” my take is that their behavior is best *not* studied phenomenologically, but at the "inference time algorithmic level."
*I agree with the second objection*: the fact that using theory-of-mind comes natural does not necessarily mean it's good. I have Opinions on its utilitarian value, but I'll keep them to myself until I can back them up properly.
You write: 'we do use theory-of-mind to understand fictional characters, but also constantly recognize that they are fictional.' To this, I have two objections. First, do we always recognize this distinction? My personal experience suggests that the boundary is more fluid. When I'm trying to understand the motives of a character in a movie, a historical figure, or a person sitting next to me, the process feels remarkably similar. The label 'fictional' doesn't seem to play a significant role in this process. Instead, I believe the key factor is interactivity, as you also point out. However, consider the world of improv theater. Here, performers — conscious agents — interact with each other and the audience to create characters — non-conscious constructs. Just like in our interactions with chatbots, we engage with these characters as if they have minds of their own, even though we know they're constructs. Role-play is another example and, as a fellow Goffman/Berne/Johnstone fan, you can probably imagine where this line of thought would bring me if I felt more flamingly inclined (:
Finally, on the utility of the friendly vs. surgical approach: while I do very much agree on the importance of studying the model at the inference time, I would need some evidence to back up the claim that the phenomenological approach is somewhat ill-advised. While I support (and do my little to contribute to) to technical, algorithmic-level understanding of the models outputs, I do fear that - past a certain level of complexity - that might lead to missing the Aunt Hillary for the drone.
I might be a bit too western-woo-tradition-oriented for your taste here, but for me "understanding a system" is a gradient that goes from being able to predict, then steer and, finally, control such system. As of now, I think most success in these endeavours has been garnered by taking the models' outputs at face value - ie, as post-author-mourning TEXT.
BTW, I was discussing this with a friend, and I suspect you should find a way to overcome your ideological differences as, in more than one occasion, I thought you might very much enjoy each other's insights. He also wrote a dialogue where (and please forgive me for being the distal cause for it) I debate a chimera of Eliezer and yourself (:
Nice read :)
(tho personally i enjoy some controversial schizo takes, or unexpected possibly true things, sprinkled with my straightforwardly true statements :p but super well-written!)
yea, this is part of my Normie Outreach Program; probably more of your liking: https://lumpenspace.substack.com/p/the-map-becomes-the-territory-wmtp?utm_source=profile&utm_medium=reader2
This is an interesting analysis! I think at the moment I disagree with it, though. Three points...
First, we do use theory-of-mind to understand fictional characters, but also constantly recognize that they are fictional. Chatbots aren’t fictional, and are interactive, so the way people relate to them is likely to be only partly analogous.
Second, there’s a descriptive vs normative issue. Although people naturally *do* use theory-of-mind to relate to chatbots, that doesn’t mean it is a good idea. The consequences of relating to them that way may be quite different from the consequences of relating to fictional characters that way.
Finally, whereas what you suggest may be true for “phenomenological study of the model's behaviour,” my take is that their behavior is best *not* studied phenomenologically, but at the "inference time algorithmic level."
*I agree with the second objection*: the fact that using theory-of-mind comes natural does not necessarily mean it's good. I have Opinions on its utilitarian value, but I'll keep them to myself until I can back them up properly.
You write: 'we do use theory-of-mind to understand fictional characters, but also constantly recognize that they are fictional.' To this, I have two objections. First, do we always recognize this distinction? My personal experience suggests that the boundary is more fluid. When I'm trying to understand the motives of a character in a movie, a historical figure, or a person sitting next to me, the process feels remarkably similar. The label 'fictional' doesn't seem to play a significant role in this process. Instead, I believe the key factor is interactivity, as you also point out. However, consider the world of improv theater. Here, performers — conscious agents — interact with each other and the audience to create characters — non-conscious constructs. Just like in our interactions with chatbots, we engage with these characters as if they have minds of their own, even though we know they're constructs. Role-play is another example and, as a fellow Goffman/Berne/Johnstone fan, you can probably imagine where this line of thought would bring me if I felt more flamingly inclined (:
Finally, on the utility of the friendly vs. surgical approach: while I do very much agree on the importance of studying the model at the inference time, I would need some evidence to back up the claim that the phenomenological approach is somewhat ill-advised. While I support (and do my little to contribute to) to technical, algorithmic-level understanding of the models outputs, I do fear that - past a certain level of complexity - that might lead to missing the Aunt Hillary for the drone.
I might be a bit too western-woo-tradition-oriented for your taste here, but for me "understanding a system" is a gradient that goes from being able to predict, then steer and, finally, control such system. As of now, I think most success in these endeavours has been garnered by taking the models' outputs at face value - ie, as post-author-mourning TEXT.
BTW, I was discussing this with a friend, and I suspect you should find a way to overcome your ideological differences as, in more than one occasion, I thought you might very much enjoy each other's insights. He also wrote a dialogue where (and please forgive me for being the distal cause for it) I debate a chimera of Eliezer and yourself (:
I'm sorry to be that person, but I don't think anyone living can really understand what anthropomorphizing humans would entail.
way to mindlessly predict the next token fren XD
Gib reward function (headpats)
I just want people to stop anthropomorphizing billionaires.
short skinsuits