Why are AI Researchers Concerned About AI Risk?

In an Op-ed from Saturday, Cal Newport dismisses catastrophic AI risk and implies that the people warning about it can be ignored as some kind of weird “Rationalists”:When Mr. Altman [CEO of OpenAI] declares their pending models will be “sobering” for humanity or Mr. Amodei [CEO of Anthropic] frets that this technology “will test who we are as a species,” it doesn’t necessarily mean that they discovered frightening new evidence that something catastrophic is imminent.It is, instead, representative of how Rationalists always think about A.I. In these circles, it’s taken for granted that A.I. capabilities will rapidly accelerate and completely transform the world, and to talk about it in any other way would be considered uninformed. For a Rationalist, the only open question is to what degree their heroic brains, carefully trained in the style glamorized by Mr. Yudkowsky’s “Sequences,” can prevent these transformations from devolving into extinction events.The op-ed doesn’t address the…