AI Alignment Considered Harmful

Many thoughtful persons who want AI to go well are preoccupied with AI alignment. This means making sure that the AI models and the systems that build on them will be aligned with humanity. Basically, that AI will be nice to us. People who think that AI systems will start self-improving and get better than us at everything tend to think that AI alignment is extra important. Super-intelligent AI could take over the world, the reasoning goes, so it’d better be morally good.How do I disagree with this? Let me count the ways.First of all, aligned to what? If you want models to be aligned to humanity, you need to assume that there’s some kind of moral theory that humanity agrees on. Obviously, that is not true. We do not agree with each other. Have you seen politics? Or religion? The theory that all religions, and other approaches to life in general, share some sort of moral core is often called moral universalism. It is false. Caste systems, LGBT rights, apostasy, the role of family,…

Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town