Unified factored representations

Unified Fractured One of the most interesting papers I’ve read in the last couple of years is Questioning Representational Optimism in Deep Learning: The Fractured Entangled Representation Hypothesis (2025).I don't know what to do with the ideas in the paper, and that's a frustration of my own making.The paper suggests: what many machine learning systems learn—“the representation”—is a tangled mess; better representations can be discovered; you don’t need more data to do this; and better representations matter. The observations in the paper go like thisYou can train a neural network to reproduce a particular image, such as a butterfly or a skull. When you look at what’s been learned, it’ll be whatever it needs to get the job done. Fine.Now, as an alternative: ask people to pick what they like from abstract images, and use evolutionary computation to “breed” neural networks that will generate new images for the next generation. Repeat that, and perhaps because we’re drawn to symmetry…