Information Theory

1 long-form post on Information Theory: machine-learning research by Taha Bouhsine, each built around live, in-browser interactive visualizations.

1 post tagged #information-theory.

Feb 23, 2026

Not All Infinities Are Equal: The Cross-Entropy Asymmetry Behind Hallucination

The singularity structure of cross-entropy is asymmetric, and that asymmetry explains LLM hallucination, the CLIP modality gap, and why contrastive losses need 32K batches.