Provable Accuracy Bounds for Hybrid Dynamical Optimization and Sampling
Burns, Matthew X., Hou, Qingyuan, Huang, Michael C.
–arXiv.org Artificial Intelligence
Analog dynamical accelerators (DXs) are a growing sub-field in computer architecture research, offering order-of-magnitude gains in power efficiency and latency over traditional digital methods in several machine learning, optimization, and sampling tasks. However, limited-capacity accelerators require hybrid analog/digital algorithms to solve real-world problems, commonly using large-neighborhood local search (LNLS) frameworks. Unlike fully digital algorithms, hybrid LNLS has no non-asymptotic convergence guarantees and no principled hyperparameter selection schemes, particularly limiting cross-device training and inference. In this work, we provide non-asymptotic convergence guarantees for hybrid LNLS by reducing to block Langevin Diffusion (BLD) algorithms. Adapting tools from classical sampling theory, we prove exponential KL-divergence convergence for randomized and cyclic block selection strategies using ideal DXs. With finite device variation, we provide explicit bounds on the 2-Wasserstein bias in terms of step duration, noise strength, and function parameters. Our BLD model provides a key link between established theory and novel computing platforms, and our theoretical results provide a closed-form expression linking device variation, algorithm hyperparameters, and performance.
arXiv.org Artificial Intelligence
Oct-8-2024
- Country:
- Asia
- Afghanistan > Parwan Province
- Charikar (0.04)
- Japan > Honshū
- Kantō > Tochigi Prefecture > Utsunomiya (0.04)
- Afghanistan > Parwan Province
- Europe
- France > Occitanie
- Haute-Garonne > Toulouse (0.04)
- Germany (0.04)
- France > Occitanie
- North America > United States
- California > Monterey County
- Monterey (0.04)
- New York
- Monroe County > Rochester (0.04)
- New York County > New York City (0.04)
- Washington > Benton County
- Richland (0.04)
- California > Monterey County
- Asia
- Genre:
- Research Report (0.65)
- Technology: