Implementing a Distributed Deep Learning Network over Spark

Deep learning is becoming an important AI paradigm for pattern recognition, image/video processing and fraud detection applications in finance. The computational complexity of a deep learning network dictates need for a distributed realization. Our intention is to parallelize the training phase of the network and consequently reduce training time. We have built the first prototype of our distributed deep learning network over Spark, which has emerged as a de-facto standard for realizing machine learning at scale. Geoffrey Hinton presented the paradigm for fast learning in a deep belief network [Hinton 2006].