In AI, general methods that scale with computation end up beating systems built on human knowledge

In artificial intelligence, general methods that make use of more computation end up winning, and by a wide margin, over systems into which someone has built what they know about the problem. As Rich Sutton argues in The Bitter Lesson, this did not happen once. It has happened again and again over seventy years of research. The reason is the price of computation. Each unit of computing costs less than the last, exponentially, yet researchers tend to work as if the available power were fixed. With fixed power, putting your knowledge of the problem into the system is almost the only way to improve, and it works: the system performs sooner, and it is satisfying to see your own understanding inside the machine. A few years later there is vastly more computation available, and that knowledge-laden system stalls while a simpler one, which only knows how to search and learn, overtakes it. Chess is the clearest case. The machine that beat Kasparov in 1997 did not understand the game the way a…