Why do OpenAI's GPT-2 weights beat mine?

When I finished my project training an LLM from scratch, I was left with a minor mystery. Why were my models worse at instruction-following than the original OpenAI GPT-2 small weights? I had an evaluation that I was running, based on the instruction fine-tuning code in chapter 7 of "Build a Large Language Model (from Scratch)". The process was to train a model on samples from the Alpaca instruction-following dataset until validation loss started rising, to use that instruction fine-tuned model to generate completions to a held-back test set, and then to use an LLM to compare the results from various different models. The details are here; let's call it the IFT eval. OpenAI's original weights for GPT-2 small consistently beat my own models, even when mine got better results than theirs on a more technical evaluation, where I just measured the cross entropy loss for each model on a held-back set of test sequences. This surprised me; I would have expected a reasonably close correlation…