Notes on AI, math, and the long road. Live slow, die whenever.
Long-form pieces on machine learning interpretability, kernel methods, contrastive learning, and the geometry of neural network representations. Most posts ship with interactive visualisations you can play with in the browser. The posts run in series that build on each other; see the map for how they connect.
New here? Two doors, either one works cold: What a Finite Kernel Buys an MLP turns a neuron into something you can look at, and Attention Is Explainable Because It Is a Kernel explains why half a transformer is readable and the other half is not. Everything else grows out of those two.
Each series is a narrative: start at part 1.
6 parts · ongoing
Where do representations live, and what makes a latent space good? Contrastive learning, the geometry of embedding spaces, and why the usual activation functions work against it.
6 parts
Attention, read as kernel regression: what the softmax is really computing, why that makes it explainable, and what happens when you make the kernel cheap.
4 parts
Once everything is a kernel, what is a weight? An interlude on RKHS foundations: where a weight lives, what it can be, and why the MLP block is a representer theorem.
9 parts · ongoing
Replace the activation with a finite, positive-definite kernel and a network becomes a list of prototypes you can read, edit by hand, and finally collapse into a single fixed-point operator.
4 parts · ongoing
A trained network carries its own kernel, and a kernel can be measured. Decompose it, cut modes out of it, count the concepts it will admit to having, and find out which parts of the design were doing the work all along.
5 parts · ongoing
Numerical analysis as an architecture catalog: skip connections as an Euler step, momentum nets as half of Newton, and conservation laws as testable predictions about hidden states.