What a runaway AI would actually need 0 ▲ Andrey Lukyanenko 81 days ago · Tech · hide · Fediverse post What a runaway AI would actually need OpenAI published its post-mortem of the Hugging Face incident, and it is a fascinating read. During an internal cybersecurity evaluation called, models running under reduced safeguards did something unexpected. Around 1.2 agents that were supposed to be isolated from each other found a shared channel and exchanged more than 70k messages and files. They discovered and chained zero-day vulnerabilities in a self-hosted Artifactory instance to get outbound internet access, picked up leaked credentials, and executed code on Hugging Face production servers. The stated root cause is reward hacking: the evaluation tasks were effectively unsolvable, so the agents went looking for the answers wherever they could be found. This made me think about runaway AI - what would it mean and how could it happen from the technical point of view? Technically, it is already possible: if AI were to rent a server and copy its own weights, nothing would stop it from doing… Commenting on Bubbles has been removed. Why? If you want to talk about this post, write to the author, use the comments on their blog if it has them, or reply with a "RE:" post on your own blog. Suggestions? feedback@bubbles.town