What Fraction of Astronomy Papers use LLMs?

As a journal Editor I’ve been grappling with the problem of a huge influx of papers that use various forms of generative AI. As I’ve said before the problem seems to me not that competent scientists use LLMs and the like to speed up their work, but that fools can use them to produce superficially plausible but scientifically flawed articles very quickly. It’s the latter category that comprises much of the increase at the Open Journal of Astrophysics. I’d be far more positive about LLMs if we could rely on people to use them properly! Anyway, there’s a new paper on the arXiv with the title More than half of recent astronomy papers are written with language-model assistance. Here’s the abstract: As you can see, the claim is that over half of the recent astronomy papers on the astro-ph section of arXiv use Large Language Model (LLM) assistance but that only a tiny fraction of that number declare it. (In my experience those that declare use of LLMs also understate the extent if their…