#interpretability
3 posts tagged #interpretability.
-
Attention is Explainable Because it is a Kernel
A reading of self-attention through kernel smoothing and RKHS.
-
Not All Infinities Are Equal
The singularity structure of cross-entropy explains hallucination, the modality gap, and why contrastive losses need such big batches.
-
Activations Are Bad for Geometry
Pointwise activations factor into the layer's Jacobian as a diagonal modulation. The same modulation that buys selectivity destroys geometric structure on the data manifold.