Sugawara, Yohei
Virtual Human Generative Model: Masked Modeling Approach for Learning Human Characteristics
Oono, Kenta, Charoenphakdee, Nontawat, Bito, Kotatsu, Gao, Zhengyan, Ota, Yoshiaki, Yamaguchi, Shoichiro, Sugawara, Yohei, Maeda, Shin-ichi, Miyoshi, Kunihiko, Saito, Yuki, Tsuda, Koki, Maruyama, Hiroshi, Hayashi, Kohei
Identifying the relationship between healthcare attributes, lifestyles, and personality is vital for understanding and improving physical and mental conditions. Machine learning approaches are promising for modeling their relationships and offering actionable suggestions. In this paper, we propose Virtual Human Generative Model (VHGM), a machine learning model for estimating attributes about healthcare, lifestyles, and personalities. VHGM is a deep generative model trained with masked modeling to learn the joint distribution of attributes conditioned on known ones. Using heterogeneous tabular datasets, VHGM learns more than 1,800 attributes efficiently. We numerically evaluate the performance of VHGM and its training techniques. As a proof-of-concept of VHGM, we present several applications demonstrating user scenarios, such as virtual measurements of healthcare attributes and hypothesis verifications of lifestyles.
Einconv: Exploring Unexplored Tensor Decompositions for Convolutional Neural Networks
Hayashi, Kohei, Yamaguchi, Taiki, Sugawara, Yohei, Maeda, Shin-ichi
Tensor decomposition methods are one of the primary approaches for model compression and fast inference of convolutional neural networks (CNNs). However, despite their potential diversity, only a few typical decompositions such as CP decomposition have been applied in practice; more importantly, no extensive comparisons have been performed between available methods. This raises the simple question of how many decompositions are possible, and which of these is the best. In this paper, we first characterize a decomposition class specific to CNNs by adopting graphical notation, which is considerably flexible. When combining with the nonlinear activations, the class includes renowned CNN modules such as depthwise separable convolution and bottleneck layer. In the experiments, we compare the tradeoff between prediction accuracy and time/space complexities by enumerating all the possible decompositions. Also, we demonstrate, using a neural architecture search, that we can find nonlinear decompositions that outperform existing decompositions.
BayesGrad: Explaining Predictions of Graph Convolutional Networks
Akita, Hirotaka, Nakago, Kosuke, Komatsu, Tomoki, Sugawara, Yohei, Maeda, Shin-ichi, Baba, Yukino, Kashima, Hisashi
Recent advances in graph convolutional networks have significantly improved the performance of chemical predictions, raising a new research question: "how do we explain the predictions of graph convolutional networks?" A possible approach to answer this question is to visualize evidence substructures responsible for the predictions. For chemical property prediction tasks, the sample size of the training data is often small and/or a label imbalance problem occurs, where a few samples belong to a single class and the majority of samples belong to the other classes. This can lead to uncertainty related to the learned parameters of the machine learning model. To address this uncertainty, we propose BayesGrad, utilizing the Bayesian predictive distribution, to define the importance of each node in an input graph, which is computed efficiently using the dropout technique. We demonstrate that BayesGrad successfully visualizes the substructures responsible for the label prediction in the artificial experiment, even when the sample size is small. Furthermore, we use a real dataset to evaluate the effectiveness of the visualization. The basic idea of BayesGrad is not limited to graph-structured data and can be applied to other data types.