Simplex ETF
The simplex equiangular tight frame: the maximally-separated configuration of class prototypes, the Welch-bound optimum, and neural collapse.
-
The Three States of Information, in JAX
A runnable companion to The Three States of Information: train tiny models in JAX and measure the three states directly: the feature-covariance spectrum collapsing from high-rank (random) to a C−1-mode frame (structured), the distributional simplicity bias that fits low-order structure first (organized), the neural-collapse simplex where class-mean cosines lock onto −1/(C−1), and the alignment/uniformity split of contrastive learning running on two separate clocks. Four live JAX visualizations, every number an eigenvalue or a loss.
-
Latent on the Spectrum, in JAX
A runnable companion to Latent on the Spectrum: build a codebook as the spectral embedding of a label kernel in JAX (classical MDS with square-root eigenvalue scaling), watch a flat spectrum become the simplex and a graded one become the horseshoe, measure kernel-target alignment, split a representation into its between-class prototype frame and within-class information spectrum, and watch neural collapse grind the information to zero.
-
Latent on the Spectrum: Why Cats Sit Closer to Dogs Than to Cars
A label-similarity kernel can be turned into a target codebook by spectral embedding: retain its leading eigenmodes, scale by their square roots, and spend a finite dimension budget. Interactive experiments move that designed geometry from a simplex toward a taxonomy, then compare it with the class-mean and within-class spectra measured in trained representations.
-
Auditing Latent Space Geometry in JAX
A runnable companion to the Welch-bound latent-space post: generate GIFs and implement the JAX metrics that tell you whether embeddings are collapsing, wasting rank, forming a simplex, or pressing against the Welch floor.
-
What Makes a Good Latent Space? The Welch Bound and the Simplex
The hidden codebook inside representation learning: why collapse happens, why opposition is a trap, why class means form a simplex, and why the Welch bound sets the best geometry when too many concepts share too few dimensions.