Neural Network Pruning: A Gentle Introduction

This post will be continuously updated from time to time. Pruning is a technique of removing unimportant parameters (weights) of a deep neural network. There are two main types of pruning techniques namely: Structured Pruning and Unstructured Pruning. Figure 1 shows a fully connected network before and after pruning one neuron. For example in Figure 1, the connection to and from the pruned neuron is removed, the resulting weight shrinks from a 3 2 matrix to a 2 2 matrix. Structured pruning can also be applied to Convolutional Neural Networks (CNN).