A Modest Proposal

Would you like to have some real impact on the tech industry? Do you have $100,000 to spend? If you answered “yes” to both questions, ask software engineering researchers (the kinds of people who participated in It Will Never Work in Theory) to design a study that companies could run internally to measure the impact that genAI adoption by programmers is having on business outcomes. Spend $50K to get expert reviews from both practitioners and (other) researchers, publish all of the proposals with the reviews, and award prizes of $25K, $15K, and $10K to the three best proposals. (If you really want to have an impact, do this in two rounds so that participants can hybridize their best ideas.) $100K feels like a lot of money… Really? Compared to what you’re spending on tokens? Does anybody actually know how to measure genAI’s impact? It’ll be interesting to find out. (After all, “yes” and “no” are equally interesting answers.) Will people write decent proposals for just a few thousand…