How Spark Illuminates Deep Learning

Data scientists everywhere are delving more deeply into deep learning (DL). If you're only skimming the surface of this trend, you might think that the Spark community, which focuses on broader applications of machine learning, is watching it all from the sidelines. Though Spark is certainly at the forefront of many innovations in machine intelligence, DL industry tools and frameworks--such as TensorFlow and Caffe--seem to be grabbing much of the limelight now. Be that as it may, Spark is playing a significant, growing, and occasionally unsung role in the DL revolution. With all of that in mind, we can track Spark's growing adoption in the DL world of deep neural nets through its incorporation into the following open-source industry initiatives, tools, frameworks, and approaches: Spark for multi-language training of DL models: Deeplearning4j (DL4J) leverages Spark clusters for fast, distributed, in-memory training of DL models that were developed Scala or Java.