SpectralShift Method Extends Context Window of Gated DeltaNet Models

A new arXiv paper proposes SpectralShift, a technique that reparameterizes the spectral properties of gated DeltaNet layers to stretch their usable context window. The authors note that linear attention layers are increasingly used in place of softmax attention for long-context work, but existing extension methods typically rely on continued pretraining without altering the layer internals. Their approach instead modifies the layers themselves, aiming to make context extension more effective.