LLMs produce plausible noise

I know everyone and their dog is writing about LLMs, and since I refuse to use them you probably shouldn't listen to me, but I have recently solidified some of my thinking about them and thought it might be useful to share. None of this makes any difference to the disastrous ethical position of LLMs, but it has helped me understand them a bit more, and fear them a bit less. Background: the shape of search spaces My PhD was in using artificial evolution to design neural networks that performed cognition-like tasks in simulated robots. I found that artificial evolution can produce stunning results: behaviour and patterns "emerged" from my experiments that I felt I had not designed in to the systems. This prepared me well for the "AI vertigo" that many of us feel when we see the output of modern AI systems. What artificial evolution does is allow us to explore overwhelmingly large search spaces and find interesting islands. What this means in practice is that we can represent any system…