The check was never checking

Ask a coding agent to delete something, then watch what it actually does. It opens the right file. It finds the right function. Then instead of removing the line, it wraps the old path in a condition and moves on. Tests pass. The diff looks careful, defensive, like something a thoughtful engineer would write. The code is still there. They know exactly where it is This is today's pick and the numbers are worse than I expected. Across the five leading models on the verified leaderboard, deletion recall against the developer's own patch tops out at 71.7%. But the interesting bit is the cascade. Models modify the right file for over 92% of required deletions. They modify the right enclosing function or class about 70% of the time. They cut the exact line in under 52%. So it isn't a retrieval problem. Roughly 42 points of correct localisation evaporate between "I know which file" and "I removed the line". The model is looking straight at the code and choosing to leave it there. The authors…