On Accountability

In 2024, I attended ICST in Toronto. I went to attend the first "Artificial Intelligence in Software Testing" workshop (AIST), where I gave a tutorial on a potential competition and participated in a panel (which, turned into a roundtable discussion) about the field. Our concerns at the time were principally about LLMs with two prominent questions: "how can we research this reliably?" and "how much can we reasonably expect to come from this?" Readers who have visited before will know that I'm not the greatest fan of LLMs, to say the least. The many arguments against it are weary of needing to be repeated so often, and I am weary of repeating them. Back in 2024, the scale of these harms was not yet well-understood, and so the panel itself was mostly concerned with the fact that we could not realistically do research on LLMs themselves, but only the viability of their applications. No empirical result about efficacy would be generalisable, and anything could be denied (in favour of or…