Researchers present AttnFuse, a domain-specific language that compiles attention implementations into fused GPU kernels. The work targets the fact that attention dominates compute and memory costs in Transformer models, while new attention variants keep proliferating. The DSL aims to let developers describe such variants and get fused kernels without hand-writing low-level GPU code.
A research paper presents a method for computing exact local responses to attention interventions in transformer models. Using the derivative of rotary position embeddings, candidate edits can be scored from a cached baseline plus a single backward pass, avoiding full recomputation. The approach aims to make attention-level analysis and editing more computationally efficient.
A new arXiv paper introduces GEAR, a method for predicting human trajectories that must account for both how individuals move and how agents influence one another. The authors note that prior work has relied on attention mechanisms, graph structures, and temporal encoders to model these dynamics, and propose shifting the emphasis from dynamic encoding to dynamic activation. The work appears as a cross-listing on arXiv's cs.AI category.
A new arXiv preprint proposes a conservative finite-volume method on fixed grids in which an attention mechanism selects upstream information to build the numerical flux. The scheme is conditioned on the CFL constraint so that a shock can propagate across multiple cells in a single update without being smeared or broken. The work sits at the intersection of numerical PDE solvers and machine learning attention architectures.
A new arXiv preprint describes a deep learning pipeline that uses 3D gait recordings to help identify autism spectrum disorder, aiming to sidestep the subjectivity and cost of standard clinical assessments. The approach adds attention mechanisms to the model and evaluates results across multiple data folds to test how stable the performance is. The work is presented as a step toward more objective, non-invasive screening tools, though the abstract only covers the motivation and method rather than deployment.
A new arXiv paper introduces Warrant, a unified approach for identifying which parts of a model's attention directly influence a given metric and for steering those contributions. The work argues that relevance and permission are distinct concepts, since an attended item may not actually support the resulting prediction. It offers a way to localize and adjust attention behavior without relying solely on relevance scores.
A new arXiv paper traces hallucinations in multimodal large language models to shifts in how information is distributed across attention components the authors call synergy heads. The researchers argue that existing mitigation techniques built on attention weights track only indirect cues and therefore miss the underlying mechanism. The findings could support more targeted methods for reducing fabricated outputs in multimodal systems.