Study Ties Data and Memory Scaling to a Single Predictive Spectrum

A new arXiv paper argues that the benefit a model gains from more data and the amount of learned memory it needs are both determined by one predictive-energy spectrum in a positive-entropy autoregressive retrieval source. In this framework, each coordinate's contribution is the product of its query probability and a per-coordinate term, linking the two resources analytically. The work offers a theoretical account of how data volume and memory capacity trade off in autoregressive prediction.

Review questions whether AI scaling results justify resource allocation

A critical review examines scaling research on systems that pair a pretrained model with retrieval, search, verification, tools, and interaction. It argues that a better score achieved with a larger budget does not by itself indicate where additional resources should be directed. The paper calls for evidence that ties scaling outcomes to specific allocation decisions.

Study Scales JugnuLM Language Models From 53M to 110M Parameters

A new arXiv paper examines how a fixed sub-150M pretraining recipe behaves when model size grows from 53.5M to 109.7M parameters. Both models use a Qwen3-style decoder with grouped-query attention, RoPE, SwiGLU, RMSNorm, QK-Norm and a z-loss, trained on FineWeb-Edu data, so only scale and depth differ. The work compares the 53M and 110M variants to isolate the effects of added capacity in this small-model regime.

Distribution-Consistent Inference for Dynamic Sparse Mixture-of-Experts

A new arXiv paper tackles a limitation in Mixture-of-Experts models, which typically activate a fixed number of experts for each input. The authors propose an inference method that remains distribution-consistent when the number of active experts varies dynamically. The approach aims to preserve efficient inference in large foundation models while allowing more flexible expert routing.

Study Argues Continuous Diffusion Can Scale Competitively for Language Modeling

A new arXiv paper revisits Plaid, a likelihood-based continuous diffusion model for text, to test the assumption that continuous diffusion scales worse than discrete alternatives. The authors report that with the right design, continuous diffusion can match discrete diffusion at scale. The work is a replacement version of a cross-listed machine learning preprint.

OpenAI outlines storage platform behind ChatGPT's 1 billion users

OpenAI published an engineering account of how its storage system, Habitat, grew from an internal Python library into a distributed platform spanning multiple regions. The company says the system now handles roughly 22 million requests per second while supporting more than 1 billion ChatGPT users. The post describes the architectural changes made to keep pace with that growth.

WHY IT MATTERS ↘As frontier model quality converges, the ability to serve billions of users at tens of millions of requests per second increasingly determines cost per interaction and uptime, making bespoke storage and serving infrastructure a competitive moat rather than a back-end detail. For practitioners, it signals that data-layer architecture—not just model design—is now a primary constraint on scaling AI products, and that OpenAI is publishing this to set expectations for what production-scale deployment requires.