Data preprocessing is a fundamental part of any machine learning project and often more time is spent on the data preparation than on the actual machine learning. While some preprocessing tasks are problem specific many others such as partitioning data into training and test folds, stratifying samples or building mini-batches are generic. The following Canonical Pipeline shows the processing steps common for deep-learning in vision. A Reader reads sample data stored in text files, Excel or Pandas tables. The Splitter then partitions data into training, validation and test folds and performs stratification if needed. Usually not all image data can be loaded into memory and a Loader loads images on demand. These images are often processed by a Transformer, for resizing, cropping or other adjustments. Furthermore, to increase the training set additional images are synthesized by randomly augmenting (flipping, rotating, …) images using an Augmenter. Efficient, GPU-based machine learning demands that image and label data are grouped in mini-batches via a Batcher before passed on to the Network for training or inference. Finally, to keep track of the training progress, usually a Logger is employed to write training losses or accuracies to a log file. Some machine learning frameworks such as Keras provide (some of) these preprocessing components hidden behind an API that considerably simplify network training if it fits the task at hand. See the following excerpt of a Keras example to train a model with augmentation. However, what if an image format, augmentation or other preprocessing capability is needed that is not provided by the API? Extending a library such as Keras or others is not trivial and the common approach is to (re)implement the required functionality – often in a quick-and-dirty fashion. But implementing a robust data pipeline that loads, transforms, augments and processes images on demand is challenging and time consuming. The following excerpt from a nuts-ml example shows a pipeline for network training, where the operator defines the flow of data. In the example above training images are augmented, pixel values re-ranged, and the samples shuffled before building batches for network training. Finally, the mean over the batch-wise training losses is computed and printed. The resulting code is more readable and can readily be modified to experiment with different preprocessing schemes. Task-specific functions can easily be implement as nuts and added to the data flow. Any machine learning library that accepts mini-batches of Numpy arrays for training or inference is compatible. For more information about nuts-ml see the Introduction and have a look at the Tutorial. Bio: Stefan Maetschke (PhD) is a research scientist at IBM Research Australia where he develops machine learning infrastructure and models for medical image analysis.

Data preprocessing is a fundamental part of any machine learning project and often more time is spent on the data preparation than on the actual machine learning. While some preprocessing tasks are problem specific many others such as partitioning data into training and test folds, stratifying samples or building mini-batches are generic. The following Canonical Pipeline shows the processing steps common for deep-learning in vision. A Reader reads sample data stored in text files, Excel or Pandas tables. The Splitter then partitions data into training, validation and test folds and performs stratification if needed. Usually not all image data can be loaded into memory and a Loader loads images on demand. These images are often processed by a Transformer, for resizing, cropping or other adjustments. Furthermore, to increase the training set additional images are synthesized by randomly augmenting (flipping, rotating, …) images using an Augmenter. Efficient, GPU-based machine learning demands that image and label data are grouped in mini-batches via a Batcher before passed on to the Network for training or inference. Finally, to keep track of the training progress, usually a Logger is employed to write training losses or accuracies to a log file. Some machine learning frameworks such as Keras provide (some of) these preprocessing components hidden behind an API that considerably simplify network training if it fits the task at hand. See the following excerpt of a Keras example to train a model with augmentation. However, what if an image format, augmentation or other preprocessing capability is needed that is not provided by the API? Extending a library such as Keras or others is not trivial and the common approach is to (re)implement the required functionality – often in a quick-and-dirty fashion. But implementing a robust data pipeline that loads, transforms, augments and processes images on demand is challenging and time consuming. The following excerpt from a nuts-ml example shows a pipeline for network training, where the operator defines the flow of data. In the example above training images are augmented, pixel values re-ranged, and the samples shuffled before building batches for network training. Finally, the mean over the batch-wise training losses is computed and printed. The resulting code is more readable and can readily be modified to experiment with different preprocessing schemes. Task-specific functions can easily be implement as nuts and added to the data flow. Any machine learning library that accepts mini-batches of Numpy arrays for training or inference is compatible. For more information about nuts-ml see the Introduction and have a look at the Tutorial. Bio: Stefan Maetschke (PhD) is a research scientist at IBM Research Australia where he develops machine learning infrastructure and models for medical image analysis.

Researchers are breaking down the barrier between people and machines by teaching computers to recognize fear. On the 4th floor of the pristine Media Lab Complex at MIT lives a Nightmare Machine. These computers earned that nickname for a reason: they have been learning how to terrify people. A series of algorithms generates disturbing and grotesque images, like movie monsters, dead people, and other things that go bump in the night. "We wanted to playfully explore how artificial intelligence (AI) can become a demon that learns how to scare you," said Pinar Yanardag Delul, one of the creators of the gore-loving computer program.

In the past years, a collection of hardware, software and online service have managed to bring changes and reforms to classrooms and teaching methods. But the true disruption of education is yet to arrive. Artificial Intelligence has proven its role as a game changing factor in an increasing number of fields, causing transformations unimaginable in the past. It's now showing glimmers of how it might forever change the learning process, one of the oldest skills that mankind has mastered. Here's how AI and its derivatives are gradually finding their way into the classroom, and beyond.

Rice University computer scientists have adapted a widely used technique for rapid data lookup to slash the amount of computation--and thus energy and time--required for deep learning, a computationally intense form of machine learning. "This applies to any deep-learning architecture, and the technique scales sublinearly, which means that the larger the deep neural network to which this is applied, the more the savings in computations there will be," said lead researcher Anshumali Shrivastava, an assistant professor of computer science at Rice. The research will be presented in August at the KDD 2017 conference in Halifax, Nova Scotia. It addresses one of the biggest issues facing tech giants like Google, Facebook and Microsoft as they race to build, train and deploy massive deep-learning networks for a growing body of products as diverse as self-driving cars, language translators and intelligent replies to emails. Shrivastava and Rice graduate student Ryan Spring have shown that techniques from "hashing," a tried-and-true data-indexing method, can be adapted to dramatically reduce the computational overhead for deep learning.

Together, the last two responses add up to 71 percent of respondents, and Accenture reports that insurers are investing in AI in several areas of the business, including distribution, claims and underwriting. They are looking to empower agents, brokers and employees to enhance the customer experience with automated personalized services, faster claims handling and individual risk-based underwriting processes, Accenture said in a statement. Separately, Carrier Management interviewed representatives of four global insurance groups--XL Catlin, Allianz, QBE and Zurich--who described some of the AI initiatives already underway at their firms. They range from condensing lengthy engineering reports for swifter underwriting, reassigning some of the claims administrative services handled by offshore humans to robots, interpreting crop risk information delivered by drones, and deciphering communications from customers with heavy accents using natural language processing. Also detailed is Zurich's use of a multilingual natural language processing product called Cogito from Expert Systems that mines complicated, voluminous claims data to rapidly provide more refined information to claims adjusters for faster decision-making.

Regression is arguably the workhorse of statistics. Despite its popularity, however, it may also be the most misunderstood. The answer might surprise you: There is no such thing as Regression. The Dependent Variable is something you want to predict or explain. In a Marketing Research context it might be Purchase Interest measured on a 0-10 rating scale.

This is a great project undertaken by Jordi Warmenhoven to implement the concepts from the book An Introduction to Statistical Learning with Applications in R by James, Witten, Hastie, Tibshirani (2013) in Python (the book has practical exercises in R, as you may have guessed). The book is freely available in as a PDF, which makes this repo even more attractive to those looking to learn.

The tech giant has announced that it will improve the security level of its email service, Gmail, by introducing a couple of new features. Again, Google is tapping from the ability of machine learning to secure Gmail services. Some of the measures the company took in such situations include rejecting such messages and informing the sender of the presence of virus in an email, and preventing users from sending messages with attachments that have been infected. In the case of infected attachments, Google prevented them from being downloaded.

Tel Aviv, Israel, Thursday 1 June 2017 – Earnix, a leading provider of analytics solutions for the financial services industry, today announced the results of a global survey of insurance executives, which shows wide adoption of Machine Learning across the globe, and the expectation that ML will bring "significant" change to the industry over the next 3-5 years. Over half (54%) of the almost 200 insurance executives surveyed said that their organization was using Machine Learning for predictive analytical modelling. Of those deploying the technology, 70% said they were using it for risk modelling; followed by demand models (45%) and fraud detection (36%). Although nascent, most companies using Machine Learning have realized measurable benefits. Over half of the respondents (57%) said that Machine Learning has made their analytical models far more accurate, which has led to better risk assessments, and ultimately better decisions.