The moral hazard of large language models

One of the most off-putting things to me about LLM-based tools is that they encourage you to treat these models like people. You talk to it like it’s sentient, and you assume it has a theory of mind, and you give it tasks like you would another person. If it doesn’t work, you have a “conversation” with it as if you’re trying to explain something to another human being, then encourage it to try again. When they do what you want, you may feel a sense of companionship or collaboration as if you just worked with a human colleague. That sort of interaction necessarily engages the “human interaction” set of skills in your brain, and it’s really easy to fall into the trap of believing that an LLM is another human being. To me, this illusion creates a moral hazard, encouraging the user to imbue the statistical autocomplete machine with human characteristics that it doesn’t deserve, and explaining its (mis)behavior in human terms. Do this often enough, and you will start thinking about the…