Paper Summary: The Matthew Effect in RL

Learning to Solve Hard Problems in RL for LLMs by Never Giving Up discusses an approach for countering what the authors call the Matthew Effect, the tendency for the LLM to improve much more on easy problems that it is already good than harder problems that have a lower solve rate. Their approach is to allocate training time dynamically based on the difficulty of the problem. Background After an LLM has been pretrained on a large corpus of supervised fine-tuning (SFT) data, it is common to post-train using reinforcement learning methods like GRPO. This allows the model to improve on tasks with verifiable rewards and even exceed the performance of the original SFT data.