It's like something from a science fiction story. (No, it's literally* something from a science fiction novel)

[* Can you use literally to describe something being derived from something even if the result is not a direct quote? Well, I just did so apparently the answer is yes.]This makes perfect sense when you think about it. Large language models are designed to generate plausible strings of text with respect to their training data. Think about all the times you've heard chatbots like Claude or ChatGPT compared to science fiction or described in sci-fi terms. If you ask an LLM what it's "thinking," it is not at all surprising that it would reference the sci-fi books and plot descriptions in its training data when coming up with a reasonable-sounding answer, particularly when you take into account the noise that comes with the feedback loops of agentic systems. From Cal Newport: [emphasis added] OpenAI was looking at these reasoning traces to find unnerving examples of plotting behavior from its “swarm.” This seems like a natural thing to do, but there are two problems with this approach:…