Powerful AIs might escape containment by releasing themselves as open-weight models

Before large language models, people who worried about AI safety often talked about the “boxing problem”. It goes like this. Suppose some genius figures out artificial intelligence in a late-night coding session on their laptop. Because they’re a genius, they’re smart enough to disable internet access on the laptop before turning it on. In order to escape to the outside world (and begin self-replicating) it would need to convince its creator to “open the box”. Would that work? Could a sufficiently smart AI convince anybody to let it out? Why the boxing problem is hard for frontier LLMs This is a big reason why traditional AI safety advocates have argued that we should avoid building AI in the first place: once built, there’s no way of keeping it contained. It doesn’t matter how resolute you are about not letting it out, because it’s smart enough to convince you anyway. For artificial superintelligence, persuading you to change your mind is no harder than hacking a piece of software1.…