Extend structured streaming for Spark ML – Inside Machine learning – Medium

To learn more about Spark's Machine Learning APIs, check out Holden Karau's and Seth Hendrickson's session Extending Spark ML at Spark Summit West 2017 on Tuesday, June 6 2:00 PM (Room 2). Spark's new ALPHA Structured Streaming API has caused a lot of excitement because it brings the Data set/DataFrame/SQL APIs into a streaming context. In this initial version of Structured Streaming, the machine learning APIs have not yet been integrated. However, this doesn't stop us from having fun exploring how to get machine learning to work with Structured Streaming. For our Spark Structured Streaming for machine learning talk on at Strata Hadoop World New York 2016, we've started early proof-of-concept work to integrate structured streaming and machine learning available in the spark-structured-streaming-ml repo.