Essay: Frontier AI Cannot Be Audited

As many of you know, I started an algorithmic auditing firm ten years ago, after publishing a book called Weapons of Math Destruction: how big data increases inequality and threatens democracy. I saw how algorithms and AI were infiltrating every human bureaucracy, making important if not life-altering decisions about our daily lives, and I wanted to set better standards in algorithmic auditing than what I’d seen during the financial crisis, where investment banks essentially bought AAA ratings on very questionable mortgage-backed securities. With a background in mathematics and data science, and with a keen interest in the public good, I was well-poised to do this work. Fast forward to now. After many years of hands-on experience auditing algorithmic systems that decide people’s insurance premiums, mortgage rates, student loan APR, hiring decisions, and online experiences with dating, socializing, and searching for homes, I’m here to report that the frontier AI models cannot be…