The Curve Bends You

The plan is no plan. That is not the worst possible plan. But it is close. Since before the transformer, we have warned that the most suicidal thing you could do would be to ask your AI to do your alignment homework, and automate the process. Alignment is complex and interacts deeply with every aspect of the world, and is one of the hardest possible things for an AI to get right even if it means maximally well and is itself functionally aligned. Mistakes get amplified up the chain, you get exactly what you optimized for, and you are rather doomed. Never go full RSI. Yet, now more than ever, this seems to be the plan: Solve prosaic issues and have operational excellence, to align current AI. Have current AI do automated alignment work and figure it all out, aka ????. Profit. The good news is, most people realize this is at best a no-good terrible plan and would prefer a better one. The bad news is that some people still think this is a plan that, while not great, still has a solid…