Collaborating Authors

HDInsight -New self-paced trainings and labs


More and more organizations are taking on the challenge of analyzing big data. This course teaches you how to use the Hadoop technologies in Microsoft Azure HDInsight to build batch processing solutions that cleanse and reshape data for analysis. In this five-week course, you'll learn how to use technologies like Hive, Pig, Oozie, and Sqoop with Hadoop in HDInsight; and how to work with HDInsight clusters from Windows, Linux, and Mac OSX client computers. Course Syllabus Module 1: Getting Started with HDInsight The course begins with an introduction to big data concepts and Hadoop, before examining Microsoft Azure HDInsight and the Hadoop distribution it provides. You'll learn how to provision an HDInsight cluster, upload data to the cluster, and perform Map/Reduce jobs that process the data.

Applying Deep Learning at Cloud Scale, with Microsoft R Server & Azure Data Lake


This post is by Max Kaznady, Data Scientist, Miguel Fierro, Data Scientist, Richin Jain, Solution Architect, T. J. Hazen, Principal Data Scientist Manager, and Tao Wu, Principal Data Scientist Manager, all at Microsoft. Today's businesses collect vast volumes of images, video, text and other types of data – data which can provide tremendous business value if efficiently processed at scale and using sophisticated machine learning algorithms. Example applications include real-time labeling and monitoring of sentiment in tweets, itemization of equipment and materials at construction sites through video surveillance, and real-time fraud detection in the financial domain, to name a few. In a previous blog post, we described how to set up DNNs in the cloud using a high performance GPU VM and MXNet. In this sequel, we outline a pipeline process for training and scoring with DNNs in a large-scale production environment.



Apache Spark enhancements are on the way, as are a whole host of new products. Microsoft's cloud-based big data offering HDInsight is being handed some new capabilities, but that's not the only thing that users will be looking forward to. The Azure hosted service, which is based off Apache Hadoop, HBase, Spark, Microsoft R Server and more, is having its pricing restructured across a number of product areas. First up is an up to 52% price reduction in Azure HDInsight, which will include the replacement of the Premium cluster tier with the Enterprise Security Package. Microsoft's bumper Christmas savings bonanza also includes an additional 80% price reduction for R Server for Azure HDInsight.