papersTODAY 04:00 UTC
AttnFuse: A Composable DSL for Compiling Attention Variants into Fused GPU Kernels
Researchers present AttnFuse, a domain-specific language that compiles attention implementations into fused GPU kernels. The work targets the fact that attention dominates compute and memory costs in Transformer models, while new attention variants keep proliferating. The DSL aims to let developers describe such variants and get fused kernels without hand-writing low-level GPU code.