Goodhart's law

Maybe reward hacking is not psychology, but maths. Optimization finds the gap between a measure and what it measures.