Deep learning with kernels through RKHM and the Perron-Frobenius operator

Reproducing kernel Hilbert C * -module (RKHM) is a generalization of reproducing kernel Hilbert space (RKHS) by means of C * -algebra, and the Perron-Frobenius operator is a linear operator related to the composition of functions. We derive a new Rademacher generalization bound in this setting and provide a theoretical interpretation of benign overfitting by means of Perron-Frobenius operators. By virtue of C * -algebra, the dependency of the bound on output dimension is milder than existing bounds. We show that C * -algebra is a suitable tool for deep learning with kernels, enabling us to take advantage of the product structure of operators and to provide a clear connection with convolutional neural networks. Our theoretical analysis provides a new lens through which one can design and analyze deep kernel methods.