Neural Path Features and Neural Path Kernel: Understanding the role of gates in deep learning Chandrashekar Lakshminarayanan and Amit Vikram Singh

A deep neural network (DNN) with ReLU activations has many gates, and the on/off status of each gate changes across input examples as well as network weights. For a given input example, only a subset of gates are active, i.e., on, and the sub-network of weights connected to these active gates is responsible for producing