Multiscale systems are ubiquitous in science and engineering. Modeling and controlling such systems is essential due to their prevalence in natural and engineered systems, and understanding their behavior requires a multidisciplinary approach that integrates models, and experimental techniques at multiple scales [1]. These complex systems generally have dynamics operating at different spatiotemporal scales, such as a fine or microscale, and a coarse or macroscale. Microscale modeling usually involves analyzing the system behavior at fine resolutions, thus entailing a substantial computational cost while capturing the system's long-term behavior. On the other hand, the macroscopic models are efficient, but their accuracy hinges on the ability to capture the system dynamics effectively. Another challenge in studying multiscale systems is that the governing equations may be explicitly known at the microscopic/individual level, but the closures required to translate them to high-level macroscopic descriptions remain elusive.

The correspondence between residual networks and dynamical systems motivates researchers to unravel the physics of ResNets with well-developed tools in numeral methods of ODE systems. The Runge-Kutta-Fehlberg method is an adaptive time stepping that renders a good trade-off between the stability and efficiency. Can we also have an adaptive time stepping for ResNets to ensure both stability and performance? In this study, we analyze the effects of time stepping on the Euler method and ResNets. We establish a stability condition for ResNets with step sizes and weight parameters, and point out the effects of step sizes on the stability and performance. Inspired by our analyses, we develop an adaptive time stepping controller that is dependent on the parameters of the current step, and aware of previous steps. The controller is jointly optimized with the network training so that variable step sizes and evolution time can be adaptively adjusted. We conduct experiments on ImageNet and CIFAR to demonstrate the effectiveness. It is shown that our proposed method is able to improve both stability and accuracy without introducing additional overhead in inference phase.