Ablation

Aug 11, 2026

I Removed Every MLP from Gemma 4 12B

Deleting every feed-forward branch removes 8.5 billion parameters and makes local Gemma 4 inference roughly three times faster. It also turns a capable language model into a machine that emits the audio control token forever.