Build a Neural Network from Scratch in 60 lines of OCaml Code

People have been asking me what is the current development state of Owl (a numerical library in OCaml). Well, I think it is about time to show what Owl can actually do at the moment with its newly added AD (Algorithmic Differentiation) module. I will demonstrate how to build a small two-layer neural network from scratch in order to learn the hand-written digits in MNIST dataset. The following code snippet defines a simple two-layer neural network with tanh and softmax as the activation function for the first and second layer respectively. Remember to open Owl and Algodiff.AD modules. Defining a network seems trivial, but how about the core component in all neural networks: back propagation?