Spark for Scale: Machine Learning for Big Data – Learning New Stuff

Recently we shared an introduction to machine learning. While making machines learn from data is fun, the data from real-world scenarios often gets out of hand if you try to implement traditional machine-learning techniques on your computer. To actually use machine learning with big data, it's crucial to learn how to deal with data that is too big to store or compute on a single computing machine. Today we will discuss fundamental concepts for working with big data using distributed computing, then introduce the tools you need to build machine learning models. We'll start with some naive methods of solving problems, which are meant only as an example. As we move forward, we will make things more realistic. MapReduce is a technique that is used to distribute a data set in parts to different agents. An agent here means a single computer.