Learning-Rate-Free Learning by D-Adaptation
Paper
•
2301.07733
•
Published
Descent Only
Papers, posts and resources related to optimization for ML.
-
-
DoG is SGD's Best Friend: A Parameter-Free Dynamic Step Size Schedule
Paper • 2302.12022 • Published • 1 -
Lion Secretly Solves Constrained Optimization: As Lyapunov Predicts
Paper • 2310.05898 • Published • 2 -
When, Why and How Much? Adaptive Learning Rate Scheduling by Refinement
Paper • 2310.07831 • Published • 1 -
Stochastic model-based minimization of weakly convex functions
Paper • 1803.06523 • Published -
Handbook of Convergence Theorems for (Stochastic) Gradient Methods
Paper • 2301.11235 • Published