Computer scientist Cal Newport adds some clarity to the AI debate

As previously mentioned, software veteran Carl Brown has pointed out that we can prevent incidents like the hugging face hack simply by enforcing existing laws.Here the reliable Cal Newport explains that these incidents all come not from LLMs in general but from the reckless application of one particular application of LLM-powered agents, and that there's no good explanation for the risks these companies have been taking. When Hubinger says “AI could kill all humans,” he’s not talking about AI in a general sense; he’s really referring to a specific type of AI system that we can call an LLM-powered agent. These systems consist of an old-fashioned computer program that repeatedly does something like the following: Sends a prompt to an LLM describing its goal and current state, then asks for a suggestion about what to do next to move closer to the goal.Blindly executes whatever the LLM describes in its response.Updates its state based on what happens and then loops back to the first…