A Gentle Introduction to Math Behind Neural Networks

Today, with open source machine learning software libraries such as TensorFlow, Keras or PyTorch we can create neural network, even with a high structural complexity, with just a few lines of code. Having said that, the Math behind neural networks is still a mystery to some of us and having the Math knowledge behind neural networks and deep learning can help us understand what's happening inside a neural network. It is also helpful in architecture selection, fine-tuning of Deep Learning models, hyperparameters tuning and optimization. I ignored understanding the Math behind neural networks and Deep Learning for a long time as I didn't have good knowledge of algebra or differential calculus. Few days ago, I decided to to start from scratch and derive the methodology and Math behind neural networks and Deep Learning, to know how and why they work.