papersTODAY 04:00 UTC
Study investigates diffusion approximations of stochastic gradient descent
A preprint examines the use of diffusion approximations, which substitute Gaussian noise for the randomness of stochastic gradients, as a way to analyze how stochastic gradient descent behaves over time. The authors probe the conditions under which such approximations remain valid for describing the algorithm's long-run dynamics.