MixedMath - explorations in math and programing

I've heard it said that reading LLM prose is a bit like remembering a dream. It can be ok if you were the one who had the dream, but all details are fuzzy and unhelpful if you try to describe it to others. Nonetheless, I think it can be useful to share LLM-generated or LLM-assisted writing with explicit AI content warnings. For now, I will give AI content warnings in approximately the following shape: AI Written (and implicitly I think you need a very good reason to share unedited AI output) AI…

You know what's great? Writing things down and sharing them. I want to do that more here. To make this happen more, I'm going to intentionally, deliberately lower the bar for what I think is suitable to put here. I might get lucky and have a quantity-breeds-quality aspect. But I also know that I've been emailing and zuliping a manner of notes that I've written over the last few years that aren't available here. Why not? Because the bar is too high! So I'm going to lower it.

At today's Paper Talk session cohosted by IDI, I'm giving a talk today on the paper "The Free Transformer" by Fleuret (preprint available here). This paper describes a modification of the standard decoder-only transformer that learns to condition token generation on random latent variables learned without supervision. We describe some of the intuition and background for the new architecture. My talk is a RevealJS presentation, and is available here. This architecture is very straightforward t...