Fraudulent

Fraudulent. That’s the word I keep coming back to when I reflect on the experience of using chatbots and agents. These machines produce an output using conversational language and are intended to simulate communication with another person. But it's a lie. What's more pernicious is that, the more you play along, the more interesting and apparently useful the output is. It's more convenient to ignore the lie. This reminds me of stolen valor, when people put on a uniform and decorate themselves as if they served bravely in the military. It’s a lie. The terms we assign these products are examples of this: assistant, coach, advisor, coworker, companion. They impose a false image of what these products are capable of. I care about truth. We are always talking about trust. How do we make these agents more trustworthy? (Set aside the fact that these machines are indifferent to reality and truth). That goal feels incompatible with the way the underlying models are designed.