#theory

40 curated events

papersTODAY 04:00 UTC

Optimal Switching Regret Bounds for Multi-Armed Bandits Against Oblivious Adversaries

This paper studies adversarial multi-armed bandit problems in which the benchmark arm sequence may change up to S times over the course of play, a setting known as switching regret. It reviews and develops regret guarantees of order the square root of (S+1)KT, which prior work showed is achievable when S is known in advance. The work aims to pin down the optimal achievable rate under an oblivious adversary.

bandits/theory/online-learning/arxiv

papersTODAY 04:00 UTC

Convergence rate analysis of generative drifting flows

A new arXiv paper examines whether drifting models, which learn a gradual transport process during training but generate samples in a single step, can converge quickly to a target distribution. The authors identify obstructions to fast convergence at fixed scale and propose a multihead approach that improves convergence rates. The work is theoretical, focused on the training dynamics rather than a deployed system.

generative-models/training/research/theory

papersTODAY 04:00 UTC

arXiv paper studies recurrent graph neural networks and set-based aggregation

A new arXiv preprint examines recurrent graph neural networks, which repeatedly run message passing until it converges. The authors note that existing logical characterizations of these models rely on multi-set aggregation, counting-based graded logics, and halting or acceptance conditions that cannot be verified from the network's parameters.

research/graph-neural-networks/arxiv/theory

papersTODAY 04:00 UTC

Explicit Solution Derived for Five-Expert Prediction PDE and COMB Optimality Set

A new preprint presents a closed-form solution to the stationary prediction-with-expert-advice partial differential equation in the case of five experts. The solution is split into three regions, with the first two given by the four-expert result plus a single integral term. The paper also characterizes the exact set on which the COMB aggregation strategy is optimal.

research/online-learning/expert-advice/theory

papersTODAY 04:00 UTC

Study Ties Augmentation Graph Structure to Contrastive Learning Approximability

A theoretical paper examines the foundations of contrastive learning, a method that uses data augmentation to learn feature representations without large labeled datasets. The authors analyze how the structure of the augmentation graph relates to whether neural networks can approximate the resulting objective. The work aims to fill gaps in the theoretical understanding of why contrastive learning works in practice.

contrastive-learning/research/theory/self-supervised

papersTODAY 04:00 UTC

arXiv Paper Proposes Positive Topology Framework Linking Models, Observable Properties

A new arXiv preprint develops a conceptual and operational framework called Positive Topology, built on a basic relation between points or models and observable properties. Two complementary structures arise from that relation, the first of which captures a notion of universal refinement. The work also discusses forcing matrices, positivity, and information as part of the account.

research/arxiv/theory/math

papersTODAY 04:00 UTC

Quantum model certification cost tied to measurement correlation, not parameter count

A new arXiv paper examines how expensive it is to certify the Fisher information geometry of a trained variational quantum model, noting that stating the required shot budget is uncommon practice. The authors argue that the dominant cost driver is correlation between measurements rather than the number of parameters in the model. The work provides a way to quantify the number of shots needed to certify an empirical Fisher matrix to a given relative Frobenius error.

quantum/variational-models/arxiv/theory

papersTODAY 04:00 UTC

Projection-Free Methods for Stochastic Constrained Compositional Optimization

A new arXiv paper develops projection-free algorithms for stochastic optimization problems whose objectives are nested compositions of smooth functions over a closed convex decision set. The work targets the multi-level compositional setting, where gradients must be estimated through several layers of functions. It aims to avoid costly projection steps while still handling constraints.

optimization/arxiv/theory/algorithms

papersTODAY 04:00 UTC

Paper Proposes Self-Certification of Representation Adequacy for Agents

A new arXiv paper examines a structural risk for agents that act on compressed summaries of their history: when the summary conflates histories that call for different optimal actions, no decision rule defined over that summary can avoid a persistent per-round loss. The authors propose sequential self-certification of representation adequacy, framed around achieving minimum task loss. The work is cross-listed in cs.AI and cs.LG.

agents/representation-learning/arxiv/theory

papersTODAY 04:00 UTC

Paper Argues Formal Language Properties Should Constrain Neural Models

A new arXiv preprint argues that current neuroscience and language-model research mostly checks whether brain signals or model layers can predict annotated linguistic variables, which shows correlation but not how language is actually implemented. The author proposes instead deriving what a neural system must be capable of from the formal properties of language itself, then treating those requirements as constraints on neural dynamics. This reframes the goal from prediction accuracy toward identifying the mechanisms a system needs in order to support language.

research/language-models/neuroscience/theory

papersTODAY 04:00 UTC

Paper Models Retrieval-Guided Fine-Tuning as a Noisy Estimation Problem

A new arXiv paper frames retrieval-guided fine-tuning as a noisy estimation problem, where retrieved documents injected into the training objective introduce statistical uncertainty. The authors derive risk bounds and analyze how architecture choices affect performance under imperfect retrieval. The work aims to give a theoretical grounding for training setups that mix retrieval with fine-tuning.

rag/fine-tuning/theory/research

papersTODAY 04:00 UTC

Online Inverse Linear Optimization Algorithm Achieves O(d) Regret

A new arXiv paper presents a deterministic algorithm for online inverse linear optimization that attains O(d) regret, a bound that holds uniformly across the time horizon. Each round requires O(d^2) computation time. The result matches a recent bound by Dewasurendra and addresses an open question raised by Gollapudi and colleagues.

optimization/algorithms/online-learning/theory

papersTODAY 04:00 UTC

Thin-shell stability yields faster logconcave sampling from a cold start

A new arXiv paper proves that logconcave probability measures lying along the Gaussian cooling path satisfy a thin-shell stability property, extending the classical thin-shell theorem. This stability result translates into better complexity bounds for the core task of drawing samples from an arbitrary logconcave distribution, including when the process begins from a cold start rather than a warm one. The work sits in the theory of Markov chain Monte Carlo and sampling algorithm analysis.

sampling/logconcave/theory/optimization

papersTODAY 04:00 UTC

Halpern Anchoring Boosts Last-Iterate Guarantees for Stochastic Variational Inequalities

A new arXiv paper studies a single-loop, single-call stochastic algorithm that uses Halpern anchoring to solve constrained convex-concave problems and monotone variational inequalities. The method draws one unbiased sample of the gradient operator per iteration. The authors report improved last-iterate convergence guarantees for this anytime setting.

optimization/variational-inequalities/stochastic-algorithms/theory

papersTODAY 04:00 UTC

Paper Proposes Representational Accessibility Framework for Neural Scaling Laws

A revised arXiv preprint introduces "coupled scaling," a framework that explains when two learning systems trained on the same task under identical resource constraints should exhibit matching scaling rates and when they should diverge. The approach ties scaling behavior to representational accessibility, describing how easily a model can reach task-relevant features.

scaling-laws/research/arxiv/theory

papersTODAY 04:00 UTC

arXiv Paper Examines Symmetries and Singularities in Over-Parameterized Neural Networks

A new arXiv preprint argues that parameter counts and Hessian rank are insufficient for measuring the effective complexity of deep neural networks, since many different parameter settings produce identical predictions. The work analyzes the symmetries and singular structure of the loss landscape to better characterize model complexity. It appears in the cs.LG category as a new submission.

deep-learning/arxiv/theory/over-parameterization

papersTODAY 04:00 UTC

Study investigates diffusion approximations of stochastic gradient descent

A preprint examines the use of diffusion approximations, which substitute Gaussian noise for the randomness of stochastic gradients, as a way to analyze how stochastic gradient descent behaves over time. The authors probe the conditions under which such approximations remain valid for describing the algorithm's long-run dynamics.

sgd/optimization/theory/arxiv

papersTODAY 04:00 UTC

Paper Argues LLMs Act as Lossy Compressors, Not Solomonoff Induction Estimators

A new arXiv paper examines the widely discussed question of whether large language models function as Solomonoff induction estimators, a topic bridging algorithmic information theory and machine learning. The authors contend that LLMs instead behave as Shannon-style lossy compressors, and they argue that major capability leaps would require symbolic model synthesis carried out in program space rather than scaling alone.

arxiv/theory/compression/llm-capabilities

papersTODAY 04:00 UTC

Paper Sets Minimax Regret Bounds for Bandits with Probing Feedback

A new arXiv paper studies a bandit setting where a learner may probe up to k of n arms per round and only observes the highest reward among those probed, rather than each individual reward. The authors derive two minimax laws characterizing when probing yields a statistical advantage over standard bandit learning, covering independent stochastic reward models. They also identify limits on what can be learned from this winner-only feedback.

research/bandits/regret-bounds/theory

papersTODAY 04:00 UTC

Paper narrows complexity gaps in nonconvex finite-sum optimization

A new arXiv paper studies finite-sum optimization under individual smoothness assumptions, where the best achievable incremental first-order oracle complexity has remained unresolved. It proposes a "dense weak hiding" approach that tightens the gap between existing algorithms, which need roughly n plus sqrt(n) times a smoothness-scaled term, and previously known lower bounds. The results cover both general nonconvex objectives and those satisfying the Polyak-Lojasiewicz condition.

optimization/theory/machine-learning/research

papersTODAY 04:00 UTC

arXiv paper links control theory, inference, transport and thermodynamics in learning

A new arXiv preprint surveys how methods for learning structure from high-dimensional data connect to ideas from control theory, statistical inference, optimal transport and thermodynamics. The author argues these shared mathematical foundations bridge physics and applied mathematics with machine learning. The paper also outlines applications of this unified perspective.

research/theory/arxiv/machine-learning

papersTODAY 04:00 UTC

KL-Regularized Contextual Bandits Achieve Logarithmic Regret via Greedy Sampling

A new arXiv paper analyzes KL-regularized contextual bandits under both reward and preference feedback. The authors show that a greedy sampling approach attains logarithmic regret without an explicit dependence on the eluder dimension. The work covers regret guarantees for the reward-feedback setting and extends the analysis to preference-based feedback.

bandits/research/theory/regret-bounds

papersTODAY 04:00 UTC

New Gap Entropy Method Nears Instance-Wise Optimal Best-Arm Identification

Researchers introduce a quantity called gap entropy for the best-arm identification problem with independent Gaussian arms, where the goal is to find the highest-mean arm using as few samples as possible at a given confidence level. They show that an algorithm based on this measure comes close to the optimal sample complexity for each individual problem instance. The work is a theoretical contribution posted to arXiv and has not yet been peer reviewed.

arxiv/bandits/sample-complexity/theory

papersTODAY 04:00 UTC

Sampled Constraints Bias Equilibria in Augmented Primal-Dual Dynamics

A new study examines how augmented primal-dual methods behave when constraint values are estimated from samples rather than known exactly. The authors show that unbiased constraint estimates can still yield a biased augmented multiplier signal, which moves the equilibrium point of the mean dynamics. The paper analyzes stability and convergence properties under this sampling-induced shift.

optimization/primal-dual/convergence/theory

papersTODAY 04:00 UTC

Paper Examines Global Convergence of PPO-Clip in Language Model Post-Training

A new arXiv paper analyzes the actor-only variants of Proximal Policy Optimization that are commonly used to post-train large language models. The authors derive non-asymptotic global convergence guarantees for the clipped PPO objective, addressing how the clipping mechanism affects optimization. The work offers theoretical grounding for a method widely deployed in practice.

reinforcement-learning/ppo/theory/llm-training

papersTODAY 04:00 UTC

Paper Links Turing Machine Structure to Singularities of Analytic Functions

A research paper develops a correspondence between the structure of Turing machines and the singularities of real analytic functions. It connects the Ehrhard-Regnier derivative from linear logic with geometric ideas from Watanabe's singular learning theory. The work suggests a mathematical bridge between computation and geometry.

research/theory/linear-logic/turing-machines

papersTODAY 04:00 UTC

ReLU Networks Shown to Approximate Smooth Functionals on Hilbert Space

A new arXiv preprint analyzes how well deep ReLU networks uniformly approximate smooth scalar-valued functionals defined on an infinite-dimensional separable Hilbert space. The author expresses functional inputs as expansions over a basis and derives error bounds, showing how the required network complexity scales with the decay of the input's coefficients. The results characterize a dimensional-decay effect along with accompanying error analysis.

research/approximation-theory/relu/theory

papersSEP 11 04:00 UTC

Silver Rate Proved Near-Optimal for Accelerated Gradient Descent

Researchers analyze how much predetermined stepsizes can speed up gradient descent in smooth convex optimization. They establish a lower bound matching the so-called silver rate, up to a doubly logarithmic correction factor. The result indicates that this rate is essentially the best achievable acceleration for the setting studied.

optimization/gradient-descent/convex-optimization/theory

papersSEP 11 04:00 UTC

arXiv paper studies Fisher-Rao gradient flows of linear programs and natural policy gradients

A revised arXiv preprint analyzes natural gradient methods built on the Fisher information matrix of the state-action distribution. The authors connect these methods to Fisher-Rao gradient flows of linear programs, aiming to clarify why Kakade-style natural policy gradient updates converge, including in regularized settings. The work is theoretical and sits at the intersection of optimization and reinforcement learning.

reinforcement-learning/optimization/arxiv/theory

papersSEP 11 04:00 UTC

Lower Bounds Derived for Shallow ReLU^k Network Approximation on the Sphere

A new arXiv paper proves two distinct lower bounds on how well shallow ReLU^k neural networks can approximate functions defined on the unit sphere. For any fixed choice of inner network parameters, the best L2 approximation error is bounded from below, with a second result addressing a related configuration-dependent setting. The work characterizes fundamental limits of shallow architectures with higher-order ReLU activations rather than proposing a new method.

theory/neural-networks/approximation/relu

papersSEP 11 04:00 UTC

Paper Proposes Theoretical Framework for Memorization in Diffusion Models

A new arXiv preprint develops a theoretical account of why diffusion models sometimes reproduce training data verbatim rather than generating novel samples. The authors propose smoothing the score function as a way to reduce this memorization effect and improve generalization. The work is presented as an explanation of the phenomenon rather than a new model release.

diffusion-models/memorization/theory/research

papersSEP 11 04:00 UTC

Paper Shows Minimax Regret Possible in Bilateral Trade with Heavy-Tailed Valuations

A new arXiv paper examines contextual bilateral trade where the posted price does not influence which valuations the seller observes. The authors prove that this action-independent feedback structure removes the polynomial adaptation penalty previously associated with heavy-tailed valuation distributions. The result yields minimax regret guarantees without requiring a variance bound.

research/arxiv/economics/theory

papersSEP 11 04:00 UTC

Study Links Zero Pattern of Design Matrix to Multiple Descent in Over-parameterized Regression

A new arXiv paper examines multiple descent phenomena in over-parameterized linear regression, a setting where prior work typically assumed independent covariates and non-degenerate covariance matrices. The authors relax both assumptions, showing that the zero pattern of the design matrix governs this behavior. The result offers a more general theoretical account of how model complexity affects prediction error.

research/regression/theory/arxiv

papersSEP 11 04:00 UTC

Test-time training shown to boost in-context learning of nonlinear functions

A new arXiv paper examines test-time training (TTT), a method where selected model parameters are updated before each prediction so the model can adapt to test data. The authors note that while TTT has had empirical success, its theoretical basis is not well understood, and their analysis focuses on how it affects in-context learning of nonlinear functions.

research/test-time-training/in-context-learning/theory

papersSEP 11 04:00 UTC

Near-optimal bounds on Lipschitz constants of deep random ReLU networks

This preprint analyzes the ℓ^p-Lipschitz constants of ReLU neural networks mapping from R^d to R when the weights are randomly initialized using a variant of the He scheme, covering p from 1 to infinity. The author derives estimates that are near-optimal, meaning the upper and lower bounds match up to constant factors. The work targets theoretical understanding of how depth and width affect the sensitivity of randomly initialized networks.

research/neural-networks/theory/lipschitz

papersSEP 10 04:00 UTC

Language Generation in the Limit: Finite Witnesses and a Separation-Width Hierarchy

A new paper gives a complete characterization of when an algorithm can output valid unseen elements of an unknown infinite language from any exhaustive stream of positive examples. The authors show generation is possible exactly when families admit finite witnesses, and they structure the problem's difficulty into a separation-width hierarchy over countable domains. The result extends classical inductive inference and language learnability theory to arbitrary families.

theory/language-generation/learning-theory

papersSEP 10 04:00 UTC

Bayes-Optimal Diagonal Regularization in Modal Inverse Problems Follows Closed-Form Power Law

A new machine learning theory paper establishes a 'diagonal saturation principle' for modal inverse problems. When truncation noise is isotropic, the optimal diagonal Tikhonov regularizer takes a closed-form power-law shape whose exponent is fixed entirely by the prior. The authors argue this explains why learned regularization converges on an analytic solution rather than a data-dependent one.

theory/inverse-problems/regularization/spectral-methods

papersSEP 10 04:00 UTC

Researchers propose probability-wave framework for adaptive agent dynamics in AGI

A new preprint introduces a mathematical approach that represents populations of interacting adaptive agents as probability waves, with a generalized behavioral intelligence equation yielding eigenmodes that can be tested empirically. The authors aim to capture how collective behavior emerges as artificial general intelligence systems adapt and interact.

agi/agents/theory/research

papersSEP 10 04:00 UTC

arXiv paper offers unifying perspective on probabilities as model predictions

A new cs.LG preprint tackles the long-standing philosophical divide between Bayesian and frequentist interpretations of probability. The authors propose framing probabilities as predictions generated by models, aiming to reconcile competing viewpoints. The work also examines under what conditions acting on probabilistic claims produces desirable outcomes.

machine-learning/statistics/theory/bayesian

papersSEP 10 04:00 UTC

Monograph Maps Connections Between Gaussian Processes and Kernel Hilbert Spaces

A newly updated arXiv monograph examines the relationship between two kernel-based machine learning traditions: probabilistic modeling with Gaussian processes and non-probabilistic methods built on reproducing kernel Hilbert spaces. The work lays out the mathematical connections and equivalences between the two approaches, providing a unified theoretical treatment of positive definite kernel techniques.

gaussian-processes/kernel-methods/theory/machine-learning