Data-Driven Impulse Response Regularization via Deep Learning

Impulse response estimation has for a long time been at the core of system identification. Up until some five to seven years ago, the generally held belief in the field was indeed that we knew all there was to know about this topic. However, the enlightening work by Pillonetto and De Nicolao [2010] changed this by showing that the estimate can in fact be improved significantly by assuming a Gaussian Process (GP) prior over the impulse response, which acts as a regularizer. This model-driven approach has since then been further refined [Pillonetto et al., 2011, Chen et al., 2012, Pillonetto et al., 2014], where the prior in this case could be interpreted to encode not only smoothness information, but also information about the exponential decay of the impulse response. In this paper we employ deep leaning (DL) to find a suitable regularizer via a method that is driven by data. Deep learning is a fairly new area of research that continues the work on neural networks from the 1990's. To get a brief, but informative, overview of the field of deep learning we recommend the paper by LeCun et al. [2015] and for a more complete snapshot of the field we refer to the monograph by Goodfel-low et al. [2016]. Deep learning has recently revolutionized several fields, including image recognition (e.g.