Can AI do history? Someone sent me a link to this paper “Can LLMs Act as Historians? Evaluating Historical Research Capabilities of LLMs via the Chinese Imperial Examination” Gao et. al. are trying to get LLM’s to demonstrate high order skills in historical reasoning, using a new benchmark ProHist-Bench. They determine, that no, they can’t. LLM’s still hallucinate, and more importantly, they answer questions wrong. My problem is not that they get questions wrong, but that the people doing this don’t seem to know what doing history is. I suppose that part of the problem is defining what “doing history” actually is. AI can make music, if you define music as orderly sounds coming out of a box. If you define music as a form of art created by people, then obviously it can’t. What is doing history? Gao et. al. set their tasks as answering questions (some “easy” and some “hard”) about the exam system in Imperial China and writing exam essays in the proper baguwen 八股文 style. Asking the AI to…
No comments yet. Log in to reply on the Fediverse. Comments will appear here.