Anthropic Just Threatened to Kill Billions of People. This Is Not Okay.

Yesterday, Jacob Coxon, an Anthropic engineer, resigned. As he ​explained on X​: “[OpenAI and Anthropic] are racing straight to self-improving superintelligence and gambling with our lives.” A former AI lab employee making these types of claims isn’t unprecedented. (I remember hearing back in 2023 about an ex-OpenAI engineer who was certain the company was just months away from runaway recursive self-improvement.) What proved more shocking were the responses that soon followed. Evan Hubinger, the head of “Alignment Science” at Anthropic, ​tweeted the followin​g: “Jacob is correct here—we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.” And he wasn’t alone. Another Anthropic engineer, Samuel Marks, ​joined the conversation​: “AI developers believe their technology could cause human…