Auto-research with codex: How I achieved a 212x Faster QR Kernel over baseline with Codex

Table of Contents Intro Why this problem is auto-research-able Learning Enough to Ask Better Questions The math, in brief Make serial work small with the help of the blocked Householder algorithm Codex-maxxing Kernel progress breakthroughs Breakthrough ideas Introducing idea diversity to escape the local maxima Implementation Hints What I could have done better Conclusion References Acknowledgements IntroGPU Mode, in collab with Core Automation, recently hosted an auto-research themed contest. The problem statement was to implement batched square compact-Householder QR factorization aka QR decomposition. I placed 12th out of 183 participants ending up with a 212x speedup over the baseline solution. This post is about how I got there. I will go through my approach, learnings, and bottlenecks I ran into during the contest. It was my first serious attempt at auto-research. Some people will call this "loop engineering", and honestly that is fine too. Note that you don't need to go through…