AI Alignment - A Global Authoritarian State

Recent discourse from major AI labs like OpenAI and Anthropic has increasingly centered on the existential threat posed by super-intelligent AI. These organizations are grappling with the daunting task of ensuring that a super-intelligence - an entity with cognitive capabilities far exceeding our own - remains “aligned” with human values. And sure, when a random job hopping employee gets prime time for claiming there's a 10% risk of human extinction within a few years what do you expect?I'm writing this post because I see lots of people saying that yes, sure, let's move forward slowly. After all, we didn't have LLMs (the “AI” in question) a few years ago so why not make some global agreement - like the Montreal Protocol on banning CFCs, which remains one of the most successful environmental treaties in history (UNEP, 2023 ), or the Non-Proliferation Treaty on nuclear weapons (State Dept, 1968 ), and be done with it? Both of these have been resounding successes, stopping the growth of…