The famous o3 "GeoGuessr" prompt did not work

In April last year, Kelsey Piper discovered that OpenAI’s o3 model was surprisingly good at figuring out where a photo was taken from. Like human “geoguessr” pros, o3 could sometimes take a nondescript photo of a beach and tell you exactly where it is. Here’s the example Kelsey gave: Several people reproduced this with good results: not a 100% success rate, but clearly far better than you’d do with a random human guess. The lesson here is that model capabilities can surprise us. The o3 model had been released for two weeks before Kelsey’s tweet without anyone noticing how good it was at geolocation. What obscure capabilities did we never find? What capabilities of current models are we missing today? Some people drew another lesson from this: that “prompt engineering” can unlock brand-new capabilities. This is because Kelsey had a magic prompt that she built over time. When o3 got something wrong, she would ask it how it could have avoided the mistake, and then included that in the…