
SAS: Simple Attention Sparsification via End-to-End Optimization of Context Ranking
NeurIPS Annual Conference on Neural Information Processing Systems, 2026
Replaces attention distillation with end-to-end context ranking under the language modeling loss, outperforming distillation baselines on reasoning, long-context and agentic tasks.



