Microsoft Releases .NET for Apache Spark 1.0

Last month, Microsoft released the first major version of .NET for Apache Spark, an open-source package that brings .NET development to the Apache Spark platform. The new release allows .NET developers to write Apache Spark applications using .NET user-defined functions, Spark SQL, and additional libraries such as Microsoft Hyperspace and ML.NET. Apache Spark is an open-source, general-purpose analytics engine for large-scale data processing, with built-in modules for streaming, SQL, machine learning, and graph processing. Initially developed by the AMPLab team at UC Berkeley, it can be used in conjunction with different data repositories, including the Hadoop Distributed File System, NoSQL databases, and relational data stores. Since all data is processed in-memory (RAM), Spark can be 100x faster than Hadoop for large-scale data processing.