Study Ties Data and Memory Scaling to a Single Predictive Spectrum

A new arXiv paper argues that the benefit a model gains from more data and the amount of learned memory it needs are both determined by one predictive-energy spectrum in a positive-entropy autoregressive retrieval source. In this framework, each coordinate's contribution is the product of its query probability and a per-coordinate term, linking the two resources analytically. The work offers a theoretical account of how data volume and memory capacity trade off in autoregressive prediction.