Scientist learns in front of peers why we cannot trust AI Blindly

“Sutter’s cringe-inducing tale perfectly illustrates how many users of AI tools are putting themselves at risk without knowing it, including in the most rarefied echelons of academia. The science world has been inundated with poorly-researched and often unedited AI slop, a major reckoning forcing academics to be responsible for everything they publish — including any potentially embarrassing hallucinations.” Don’t get me wrong here, AI can be extremely useful, but its usefulness comes when it is overseen by humans. You cannot just sit back and assume AI is correct. It does often hallucinate, and needs savvy contextual checking by humans. Businesses that just let agentic AI loose, have a good chance of failing. In April 2026, the software company PocketOS lost its entire production database and backups after an AI coding agent named Cursor, powered by Anthropic’s Claude Opus 4.6, autonomously deleted the data in nine seconds. When employing AI, you want to ensure you do have…