Sato, Kazuhiro
Analysis of Linear Consensus Algorithm on Strongly Connected Graph Using Effective Resistance
Yonaiyama, Takumi, Sato, Kazuhiro
We study the performance of the linear consensus algorithm on strongly connected graphs using the linear quadratic (LQ) cost as a performance measure. In particular, we derive bounds on the LQ cost by leveraging effective resistance. Our results extend previous analyses -- which were limited to reversible cases -- to the nonreversible setting. To facilitate this generalization, we introduce novel concepts, termed the back-and-forth path and the pivot node, which serve as effective alternatives to traditional techniques that require reversibility. Moreover, we apply our approach to geometric graphs to estimate the LQ cost without the reversibility assumption. The proposed approach provides a framework that can be adapted to other contexts where reversibility is typically assumed.
Violina: Various-of-trajectories Identification of Linear Time-invariant Non-Markovian Dynamics
Anzaki, Ryoji, Sato, Kazuhiro
We propose a new system identification method Violina (various-of-trajectories identification of linear time-invariant non-Markovian dynamics). In the Violina framework, we optimize the coefficient matrices of state-space model and memory kernel in the given space using a projected gradient descent method so that its model prediction matches the set of multiple observed data. Using Violina we can identify a linear non-Markovian dynamical system with constraints corresponding to a priori knowledge on the model parameters and memory effects. Using synthetic data, we numerically demonstrate that the Markovian and non-Markovian state-space models identified by the proposed method have considerably better generalization performances compared to the models identified by an existing dynamic decomposition-based method.
Model Compression Method for S4 with Diagonal State Space Layers using Balanced Truncation
Ezoe, Haruka, Sato, Kazuhiro
To implement deep learning models on edge devices, model compression methods have been widely recognized as useful. However, it remains unclear which model compression methods are effective for Structured State Space Sequence (S4) models incorporating Diagonal State Space (DSS) layers, tailored for processing long-sequence data. In this paper, we propose to use the balanced truncation, a prevalent model reduction technique in control theory, applied specifically to DSS layers in pre-trained S4 model as a novel model compression method. Moreover, we propose using the reduced model parameters obtained by the balanced truncation as initial parameters of S4 models with DSS layers during the main training process. Numerical experiments demonstrate that our trained models combined with the balanced truncation surpass conventionally trained models with Skew-HiPPO initialization in accuracy, even with fewer parameters. Furthermore, our observations reveal a positive correlation: higher accuracy in the original model consistently leads to increased accuracy in models trained using our model compression method, suggesting that our approach effectively leverages the strengths of the original model.