Becoming a benchmark

[This is a guest post by Talia Ringer. This blog post was initially written in a different file format and converted using AI. — T.] When I was wrapping up graduate school in computer science in Spring 2021, I was given access to a curious little programming model on OpenAI’s “playground.” The model, called Codex, took in natural language text and generated programs from that text. The task of automatically generating programs given an expression of programmer intent had been one of many major questions in my field of study—programming languages—for decades. This was the first time I had seen a neural model, with little to no input from anyone in my field, actually succeed at that task to any degree. This was more than a year before ChatGPT was released, and yet I knew that everything was about to change. On one hand, I was glad that programming was becoming more accessible. My mom came to visit shortly after that, and I pulled out my laptop and told her she could now program. I had…