Processing of missing data by neural networks

Learning from incomplete data has been recognized as one of the fundamental challenges in machine learning [1]. Due to the great interest in deep learning in the last decade, it is especially important to establish unified tools for practitioners to process missing data with arbitrary neural networks. In this paper, we introduce a general, theoretically justified methodology for feeding neural networks with missing data. Our idea is to model the uncertainty on missing attributes by probability density functions, which eliminates the need of direct completion (imputation) by single values. In consequence, every missing data point is identified with parametric density, e.g. GMM, which is trained together with remaining network parameters. To process this probabilistic representation by neural network, we generalize the neuron's response at the first hidden layer by taking its expected value (Section 3).