papersSEP 10 04:00 UTC
Edge-of-chaos initialization fails for higher input derivatives in wide networks
New research indicates that while the edge-of-chaos initialization scheme keeps first-order input perturbations stable in very wide randomly initialized networks, higher-order input derivatives become unstable under the same setup. Because techniques such as physics-informed losses, score matching, and derivative regularization rely on those higher derivatives, the results expose a gap in how such networks should be initialized for derivative-based training. The analysis focuses on smooth fully connected networks with scalar inputs.
Physics-informed neural networksderivative regularizationedge-of-chaos initializationhigher-order derivativesscore matchingwide neural networks
COVERAGE · 2 REPORTS · LINKS GO TO THE ORIGINAL OUTLETS
arXiv cs.AICritical initialization destabilizes higher input derivatives in wide scalar-input networks ↗SEP 10 04:00 UTC
arXiv cs.LGCritical initialization destabilizes higher input derivatives in wide scalar-input networks ↗SEP 10 04:00 UTC