If You Think You’re Refereeing an AI-Written Submission, What Should You Do?

You’re refereeing a paper and you come to suspect it was substantially and/or illicily written by an LLM. What should you do? My suggestion: Keep in mind the possibility that you are mistaken as well as the possibility that the AI use falls within the journal’s rules and was disclosed during the submission process. Pause your work refereeing the piece, as your suspicions of illicit AI use may unduly negatively influence your assessment of the submission. Gather a few examples of passages that strike you as AI-written, briefly explaining why. (Do not feed the manuscript into an AI-detector, for two reasons: first, doing so may violate the norms of confidentiality referees are expected to abide by, and second, such detectors appear to currently be of questionable accuracy.) Write to the journal’s editor or managing editor expressing your concerns and sharing the evidence you’ve gathered. With luck, the editor will look into the matter and either agree with you and relieve you of…