Thuwal
Shadowheart SGD: Distributed Asynchronous SGD with Optimal Time Complexity Under Arbitrary Computation and Communication Heterogeneity
We consider nonconvex stochastic optimization problems in the asynchronous centralized distributed setup where the communication times from workers to a server can not be ignored, and the computation and communication times are potentially different for all workers.
Newton Informed Neural Operator for Solving Nonlinear Partial Differential Equations
These methods can be broadly categorized into two types: function learning and operator learning approaches. In function learning, the goal is to directly learn the solution.
Improving the Worst-Case Bidirectional Communication Complexity for Nonconvex Distributed Optimization under Function Similarity
Effective communication between the server and workers plays a key role in distributed optimization. In this paper, we focus on optimizing communication, uncovering inefficiencies in prevalent downlink compression approaches.
Freya PAGE: First Optimal Time Complexity for Large-Scale Nonconvex Finite-Sum Optimization with Heterogeneous Asynchronous Computations
(Theorem 4). We also consider Assumption 3. Note that this assumption does not restrict the class of considered Iterative algorithms are traditionally evaluated based on their gradient complexity .
Stochastic Spectral and Conjugate Descent Methods
Dmitry Kovalev, Peter Richtarik, Eduard Gorbunov, Elnur Gasanov
Neural Information Processing Systems http://nips.cc/
Accelerated Stochastic Matrix Inversion: General Theory and Speeding up BFGS Rules for Faster Second-Order Optimization
Robert Gower, Filip Hanzely, Peter Richtarik, Sebastian U. Stich
Neural Information Processing Systems http://nips.cc/
