These days deep neural networks are ubiquitously used in a wide range of tasks, from image classification and machine translation to face identification and self-driving cars. In many applications, a single model error can lead to devastating financial, reputational and even life-threatening consequences. Therefore, it is crucially important to correct model mistakes quickly as they appear. In this work, we investigate the problem of neural network editing $-$ how one can efficiently patch a mistake of the model on a particular sample, without influencing the model behavior on other samples. Namely, we propose Editable Training, a model-agnostic training technique that encourages fast editing of the trained model. We empirically demonstrate the effectiveness of this method on large-scale image classification and machine translation tasks.

Deep Learning is a computational heavy process. Cutting down the costs is one major challenge along with data curation. Their power hungry training processes had garnered such reputation that researchers have published works reporting the carbon footprint of training networks. As things can only get complicated from here on, heading into a future with a deluge of machine learning applications, we see new strategies being invented to make training neural networks as efficient as problem. Updating a neural network to change its predictions on a single input can decrease performance across other inputs. While being simple, this approach is not robust to minor changes in the input.