PySpark Neural Network from Scratch

This article is not intended to provide mathematical explanations of neural networks, but only to explain how to apply the mathematical equations to run it using Spark (MapReduce) logic in Python. For simplicity, this implementation only uses RDDs (and no DataFrames). Similarly, I assume that you have Spark installed in your machine and that you can either run a spark-submit or a PySpark Jupyter-Notebook. All the code provided in this tutorial is available on this GitHub Repository. Also, throughout this article, I will base my explanation on one of my previous medium articles that explains the math behind a 3-layer neural network.