Motivated reasoning

Most of my concerns about AI are probably irrelevant, but what if one of them is not? At the intersection of psychology, neuroscience, epistemology, and political science, there's a concept called motivated reasoning. In short, it describes the tendency to arrive at desired conclusions by reasoning processes heavily influenced by individual motivations. An example is a person who finds reasons to keep smoking that convinces him- or herself: Perhaps the smoker gloms onto evidence that smoking reduces appetite, and might therefore reason that it's better to keep smoking, because quitting would entail a weight gain, which is unhealthy. As the example demonstrates, the reasoning process may not be particularly rigorous. While it may convince the person doing the reasoning, it convinces few other people. The process if often subconscious. The person doing the reasoning may not be aware of the predilection leading to a desirable outcome. We all engage in motivated reasoning, so it's a kind…