Spark Technology Center

Now that the dust has settled on Apache Spark 2.0, the community has a chance to catch its collective breath and reflect a little on what was achieved for the largest and most complex release in the project's history. One of the main goals of the machine learning team here at the Spark Technology Center is to continue to evolve Apache Spark as the foundation for end-to-end, continuous, intelligent enterprise applications. With that in mind, we'll briefly mention some of the major new features in the 2.0 release in Spark's machine-learning library, MLlib, as well as a few important changes beneath the surface. Finally, we'll cast our minds forward to what may lie ahead for version 2.1 and beyond. For MLlib, there were a few major highlights in Spark 2.0: While these have already been well covered elsewhere, the STC team has worked hard to help make these initiatives a reality -- congratulations!