In this course, we will examine in detail the R software, which is the most popular statistical programming language of recent years. You will start with exploring different learning methods, clustering, classification, model evaluation methods and performance metrics. From there, you will dive into the general structure of the clustering algorithms and develop applications in the R environment by using clustering and classification algorithms for real-life problems Next, you will learn to use general definitions about artificial neural networks, and the concept of deep learning will be introduced. The elements of deep learning neural networks, types of deep learning networks, frameworks used for deep learning applications will be addressed and applications will be done with R TensorFlow package. Finally, you will dive into developing machine learning applications with SparkR, and learn to make distributed jobs on SparkR.

R is ubiquitous in the machine learning community. Its ecosystem of more than 8,000 packages makes it the Swiss Army knife of modeling applications. Similarly, Apache Spark has rapidly become the big data platform of choice for data scientists. Its ability to perform calculations relatively quickly (due to features like in-memory caching) makes it ideal for interactive tasks--such as exploratory data analysis. R (SparkR) is the latest addition and support for it certainly lags the other three languages. In Spark 1.x there was no support for accessing the Spark ML (machine learning) libraries from R. The performance of R code on Spark was also considerably worse than could be achieved using, say, Scala.

Christopher Crosbie is a Healthcare and Life Science Solutions Architect with Amazon Web Services. This post is co-authored by Gopal Wunnava, a Senior Consultant with AWS Professional Services. SparkR is an R package that allows you to integrate complex statistical analysis with large datasets. In this blog post, we introduce you running R with the Apache SparkR project on Amazon EMR. The diagram of SparkR below is provided as a reference, but this video provides an overview of what is depicted.

I am excited to announce that the upcoming Apache Spark 1.4 release will include SparkR, an R package that allows data scientists to analyze large datasets and interactively run jobs on them from the R shell. R is a popular statistical programming language with a number of extensions that support data processing and machine learning tasks. However, interactive data analysis in R is usually limited as the runtime is single-threaded and can only process data sets that fit in a single machine's memory. SparkR, an R package initially developed at the AMPLab, provides an R frontend to Apache Spark and using Spark's distributed computation engine allows us to run large scale data analysis from the R shell. The SparkR project was initially started in the AMPLab as an effort to explore different techniques to integrate the usability of R with the scalability of Spark.

Creating and training machine-learning models is more complex on distributed systems, but there are lots of frameworks for abstracting that complexity. There are more options now than ever from proven open source projects for doing distributed analytics, with Python and R become increasingly popular. In this post, you'll learn the options for setting up a simple read-eval-print (REPL) environment with Python and R within the Cloudera QuickStart VM using APIs for two of the most popular cluster computing frameworks: Apache Spark (with MLlib) and H2O (from the company with the same name). To compare these approaches, you'll train a linear regression against a data set with known coefficients. Spark includes PySpark (supported by Cloudera), the Python API for Spark.

SparkR is an R package that provides a light-weight frontend to use Apache Spark from R. In Spark 1.6.0, SparkR provides a distributed data frame implementation that supports operations like selection, filtering, aggregation etc. (similar to R data frames, dplyr) but on large datasets. SparkR also supports distributed machine learning using MLlib. A DataFrame is a distributed collection of data organized into named columns. It is conceptually equivalent to a table in a relational database or a data frame in R, but with richer optimizations under the hood.