Learning Deep Analysis Dictionaries -- Part II: Convolutional Dictionaries

--In this paper, we introduce a Deep Convolutional Analysis Dictionary Model (DeepCAM) by learning convolutional dictionaries instead of unstructured dictionaries as in the case of deep analysis dictionary model introduced in the companion paper . Convolutional dictionaries are more suitable for processing high-dimensional signals like for example images and have only a small number of free parameters. By exploiting the properties of a convolutional dictionary, we present an efficient convolutional analysis dictionary learning approach. A L-layer DeepCAM consists of L layers of convolutional analysis dictionary and element-wise soft-thresholding pairs and a single layer of convolutional synthesis dictionary. Similar to DeepAM, each convolutional analysis dictionary is composed of a convolutional Information Preserving Analysis Dictionary (IPAD) and a con-volutional Clustering Analysis Dictionary (CAD). The IPAD and the CAD are learned using variations of the proposed learning algorithm. We demonstrate that DeepCAM is an effective multi-layer convolutional model and, on single image super-resolution, achieves performance comparable with other methods while also showing good generalization capabilities. ONVOLUTIONAL dictionary learning has attracted increasing interests in signal and image processing communities as it leads to a more elegant framework for high-dimensional signal analysis. An advantage of convolutional dictionaries [1]-[13] is that they can take the high-dimensional signal as input for sparse representation and processing, whereas traditional approaches [14]-[20] have to divide the high-dimensional signal into overlapping low-dimensional patches and perform sparse representation on each patch independently. It is a structured dictionary and can be represented as a concatenation of Toeplitz matrices where each Toeplitz matrix is constructed using the taps of a filter and the usual assumption is that the filters are with compact support. So a convolutional dictionary is effective for processing high-dimensional signals while also restraining the number of free parameters. To achieve efficient convolutional dictionary learning, the convolutional dictionary is usually modelled as a concatenation of circulant matrices [1]-[6] by assuming a periodic boundary condition on the signals. As all circulant matrices share the same set of eigenvectors which is the Discrete Fourier Transform (DFT) matrix, a circular convolution can be therefore represented as a multiplication in Fourier domain and can be efficiently implemented using Fast Fourier Transform (FFT). However, using a circulant matrix to approximate a general Toeplitz matrix may lead to boundary artifacts [3], [21], [22] especially when the boundary region is large. A multi-layer convolutional dictionary model is able to represent multiple levels of abstraction of the input signal.