Handwritten Digit Recognition with a Committee of Deep Neural Nets on GPUs

Current automatic handwriting recognition algorithms are already pretty good at learning to recognize handwritten digits. More than a decade ago, Multilayer Perceptrons or MLPs (Werbos, 1974; LeCun, 1985; Rumelhart et al., 1986) were among the first classifiers tested on the now famous MNIST handwritten digit recognition benchmark. Most had few layers or few artificial neurons (units) per layer (LeCun et al., 1998), but apparently back then these were the biggest feasible MLPs, trained when CPU cores were at least 20 times slower than today. A more recent MLP with a single hidden layer of 800 units achieved 0.70% error (Simard et al., 2003). The latest substantial improvement by others occurred in 2003 (Simard et al., 2003) (error rate 0.4%).