Data leakage and independence

Derek Lowe, writing on models for protein-ligand interaction Ideally, you train your model on a big pile o’ protein data, while holding out a good piece of it to keep it out of the training set. Then once your software has machine-learned its way to happiness, you see how it performs on those cases that it’s never seen before. But here’s the problem: it is very hard to set this up in a way so that the program is really seeing things that it hasn’t seen.