The problem of jagged intelligence continues a challenge for transformer-based AI. Generative Pre-trained Transformer...

The problem of jagged intelligence continues a challenge for transformer-based AI. Generative Pre-trained Transformer (GPT) models hallucinate, struggle with memory-intensive tasks, consume enormous compute and energy resources, and at times behave unpredictably. Researchers have a name for the resulting unevenness: jagged intelligence. Systems solve graduate-level problems and then stumble over something a child would get right. Beyond LLMs: A Post-Transformer World Emerges, by Samuel Greengard. The field has come so far and current LLMs really are impressive. On the other hand, this will be looked back to as “the early days.”