papersTODAY 04:00 UTC
Coresets for Softmax Attention: Improved Bounds and Efficient Constructions
A new arXiv paper studies query-oblivious coresets for softmax attention heads, defined as subsets of key-value pairs that approximate the full attention output within an error bound for every query in a ball. The authors build on earlier work by Liberty, Andoni and Kleiner on unweighted coresets to derive tighter bounds and faster construction methods. The goal is to reduce the compute and memory needed when evaluating attention over long key-value sets.