LLMs and self-referentiality

I woke up yesterday with the following thoughts, which are probably either obvious or dumb. A central thesis that many readers, including me, took from Douglas Hofstadter’s Gödel Escher Bach when young was that the secret of intelligence (and therefore, of AI) was going to have a lot to do with self-referentiality and “strange loops.” Even Roger Penrose’s The Emperor’s New Mind, which in some ways was the anti-GEB, ironically agreed with GEB about the fundamental importance of self-reference to the success or failure of the whole AI project. It claimed (incorrectly, in my view and in most experts’) that AI could never work because there was something about Gödel’s Theorem and self-reference that no computer program could ever capture, but that could be captured by exotic physics accessible to the human brain. Now, in 2026, we’ve succeeded at building AIs that outperform most humans at most intellectual tasks that are well-defined enough to judge. And at no point in the tech stack of…