The New York City taxi dataset is one of the largest publicly available datasets. It has about 1.1 billion taxi rides in New York City. Previously this dataset was explored and visualized in a number of blog posts, where the authors used various technologies (e.g., PostgreSQL and Apache Elastic Search). Moreoever, in a recent blog post our colleagues showed how to build machine learning models over one year of this dataset using Microsoft R Server (MRS) running in a 4-node Hadoop cluster. In this blog post we will use a single commodity machine to show an end-to-end process of downloading and cleaning 4 years of data, as well building and evaluating a machine learning model.

The New York City taxi dataset is one of the largest publicly available datasets, with information about 1.1 billion NYC taxi rides. This dataset has been explored and visualized in a number of blog posts, using a variety of techniques and technologies (e.g., PostgreSQL, Apache Elastic Search). A recent blog post showed how to build ML models over one years' worth of this dataset using MRS running in a 4-node Hadoop cluster. In a new blog post, Microsoft Data Scientist Dmitry Pechyoni shows us how to build a binary classification model that will predict if a passenger will pay a tip. Dmitry was able to use Microsoft R Server (MRS) to drive the entire process of building and evaluating machine learning models over hundreds of millions of examples using a single commodity machine.