Sparse Autoencoders Exhibit Compositional Behavior
| Source: HF Papers | Original article
Researchers find that sparse autoencoder features often fail to combine additively and introduce SAE Additivity Residuals (SAR) to quantify the resulting behavioral variation.
Sparse autoencoders (SAEs) have become a staple for probing the inner workings of large language models, translating dense activation patterns into a handful of linearly decoded, sparse features. A new study from the University of Wisconsin‑Madison, however, shows that the behavioral impact of those features does not always add up in a simple, additive way. The researchers introduce SAE Additivity Residuals (SAR), a metric that quantifies the portion of model behavior left unexplained after accounting for individual feature effects up to a chosen interaction order.
By sampling independent feature subsets, the authors demonstrate that population‑level SAR corresponds exactly to a normalized energy term, providing a clean theoretical grounding. Crucially, when they compare learned SAE coordinates to sum‑preserving random rotations of the same subspace, the learned representations consistently exhibit lower first‑order residuals across 18 model‑behavior configurations. This suggests that SAEs capture genuine compositional structure rather than merely reflecting downstream computation.
The finding matters because it refines how researchers interpret SAE‑derived features. If interactions between features drive behavior, tools that assume pure additivity may miss key dynamics, potentially skewing diagnostics, safety checks, or model‑editing interventions. SAR offers a systematic way to detect and quantify those interactions, sharpening the fidelity of interpretability pipelines.
Going forward, the community will likely test SAR on a broader suite of models and tasks, integrate it into existing SAE analysis frameworks, and explore whether reducing additivity residuals can improve downstream applications such as controllable generation or model debugging. As we reported on the rise of sparse attention mechanisms earlier this month, SAR adds another layer to the evolving toolbox for making large models more transparent and reliable.
Sources
Back to AIPULSEN