A new arXiv paper addresses how large language models tend to ignore information placed in the middle of long contexts, a problem studied mostly for retrieval tasks rather than long-input-to-long-output generation. The authors introduce a synthetic dataset and evaluation framework for this setting and propose a mitigation approach. The work is a revised cross-listing (v2) on arXiv cs.AI.
A new arXiv paper proposes Pull, a session router that keeps conversation history in an addressable store instead of injecting all prior turns into the context window. Rather than summarizing or truncating old dialogue, the system materializes only the historical state needed at each step, aiming to avoid the quadratic token growth of full-context prompting without permanently losing information. The approach targets multi-hundred-turn, stateful LLM sessions.
A new arXiv paper proposes SpectralShift, a technique that reparameterizes the spectral properties of gated DeltaNet layers to stretch their usable context window. The authors note that linear attention layers are increasingly used in place of softmax attention for long-context work, but existing extension methods typically rely on continued pretraining without altering the layer internals. Their approach instead modifies the layers themselves, aiming to make context extension more effective.