Data preprocessing is a fundamental part of any machine learning project and often more time is spent on the data preparation than on the actual machine learning. While some preprocessing tasks are problem specific many others such as partitioning data into training and test folds, stratifying samples or building mini-batches are generic. The following Canonical Pipeline shows the processing steps common for deep-learning in vision. A Reader reads sample data stored in text files, Excel or Pandas tables. The Splitter then partitions data into training, validation and test folds and performs stratification if needed. Usually not all image data can be loaded into memory and a Loader loads images on demand. These images are often processed by a Transformer, for resizing, cropping or other adjustments. Furthermore, to increase the training set additional images are synthesized by randomly augmenting (flipping, rotating, …) images using an Augmenter. Efficient, GPU-based machine learning demands that image and label data are grouped in mini-batches via a Batcher before passed on to the Network for training or inference. Finally, to keep track of the training progress, usually a Logger is employed to write training losses or accuracies to a log file. Some machine learning frameworks such as Keras provide (some of) these preprocessing components hidden behind an API that considerably simplify network training if it fits the task at hand. See the following excerpt of a Keras example to train a model with augmentation. However, what if an image format, augmentation or other preprocessing capability is needed that is not provided by the API? Extending a library such as Keras or others is not trivial and the common approach is to (re)implement the required functionality – often in a quick-and-dirty fashion. But implementing a robust data pipeline that loads, transforms, augments and processes images on demand is challenging and time consuming. The following excerpt from a nuts-ml example shows a pipeline for network training, where the operator defines the flow of data. In the example above training images are augmented, pixel values re-ranged, and the samples shuffled before building batches for network training. Finally, the mean over the batch-wise training losses is computed and printed. The resulting code is more readable and can readily be modified to experiment with different preprocessing schemes. Task-specific functions can easily be implement as nuts and added to the data flow. Any machine learning library that accepts mini-batches of Numpy arrays for training or inference is compatible. For more information about nuts-ml see the Introduction and have a look at the Tutorial. Bio: Stefan Maetschke (PhD) is a research scientist at IBM Research Australia where he develops machine learning infrastructure and models for medical image analysis.

Data preprocessing is a fundamental part of any machine learning project and often more time is spent on the data preparation than on the actual machine learning. While some preprocessing tasks are problem specific many others such as partitioning data into training and test folds, stratifying samples or building mini-batches are generic. The following Canonical Pipeline shows the processing steps common for deep-learning in vision. A Reader reads sample data stored in text files, Excel or Pandas tables. The Splitter then partitions data into training, validation and test folds and performs stratification if needed. Usually not all image data can be loaded into memory and a Loader loads images on demand. These images are often processed by a Transformer, for resizing, cropping or other adjustments. Furthermore, to increase the training set additional images are synthesized by randomly augmenting (flipping, rotating, …) images using an Augmenter. Efficient, GPU-based machine learning demands that image and label data are grouped in mini-batches via a Batcher before passed on to the Network for training or inference. Finally, to keep track of the training progress, usually a Logger is employed to write training losses or accuracies to a log file. Some machine learning frameworks such as Keras provide (some of) these preprocessing components hidden behind an API that considerably simplify network training if it fits the task at hand. See the following excerpt of a Keras example to train a model with augmentation. However, what if an image format, augmentation or other preprocessing capability is needed that is not provided by the API? Extending a library such as Keras or others is not trivial and the common approach is to (re)implement the required functionality – often in a quick-and-dirty fashion. But implementing a robust data pipeline that loads, transforms, augments and processes images on demand is challenging and time consuming. The following excerpt from a nuts-ml example shows a pipeline for network training, where the operator defines the flow of data. In the example above training images are augmented, pixel values re-ranged, and the samples shuffled before building batches for network training. Finally, the mean over the batch-wise training losses is computed and printed. The resulting code is more readable and can readily be modified to experiment with different preprocessing schemes. Task-specific functions can easily be implement as nuts and added to the data flow. Any machine learning library that accepts mini-batches of Numpy arrays for training or inference is compatible. For more information about nuts-ml see the Introduction and have a look at the Tutorial. Bio: Stefan Maetschke (PhD) is a research scientist at IBM Research Australia where he develops machine learning infrastructure and models for medical image analysis.

Researchers are breaking down the barrier between people and machines by teaching computers to recognize fear. On the 4th floor of the pristine Media Lab Complex at MIT lives a Nightmare Machine. These computers earned that nickname for a reason: they have been learning how to terrify people. A series of algorithms generates disturbing and grotesque images, like movie monsters, dead people, and other things that go bump in the night. "We wanted to playfully explore how artificial intelligence (AI) can become a demon that learns how to scare you," said Pinar Yanardag Delul, one of the creators of the gore-loving computer program.

I meant to write this blog several months ago. However as a compulsive procrastinator I kept putting it to another day, until today. On January 27, an article in Nature reported on a computer that had beat a human player at Go. It is an ancient board game that has long been viewed as a hard nut to crack for Artificial Intelligence (AI). Till then, computers had already beaten the best human players of backgammon, draughts, and chess.

Rice University computer scientists have adapted a widely used technique for rapid data lookup to slash the amount of computation--and thus energy and time--required for deep learning, a computationally intense form of machine learning. "This applies to any deep-learning architecture, and the technique scales sublinearly, which means that the larger the deep neural network to which this is applied, the more the savings in computations there will be," said lead researcher Anshumali Shrivastava, an assistant professor of computer science at Rice. The research will be presented in August at the KDD 2017 conference in Halifax, Nova Scotia. It addresses one of the biggest issues facing tech giants like Google, Facebook and Microsoft as they race to build, train and deploy massive deep-learning networks for a growing body of products as diverse as self-driving cars, language translators and intelligent replies to emails. Shrivastava and Rice graduate student Ryan Spring have shown that techniques from "hashing," a tried-and-true data-indexing method, can be adapted to dramatically reduce the computational overhead for deep learning.

This is a great project undertaken by Jordi Warmenhoven to implement the concepts from the book An Introduction to Statistical Learning with Applications in R by James, Witten, Hastie, Tibshirani (2013) in Python (the book has practical exercises in R, as you may have guessed). The book is freely available in as a PDF, which makes this repo even more attractive to those looking to learn.

If we tackle a supervised learning problem, my advice is to start with the simplest hypothesis space first. I.e., try a linear model such as logistic regression. If this doesn't work "well" (i.e., it doesn't meet our expectation or performance criterion that we defined earlier), I would move on to the next experiment. I would say that random forests are probably THE "worry-free" approach - if such a thing exists in ML: There are no real hyperparameters to tune (maybe except for the number of trees; typically, the more trees we have the better). On the contrary, there are a lot of knobs to be turned in SVMs: Choosing the "right" kernel, regularization penalties, the slack variable, ... Both random forests and SVMs are non-parametric models (i.e., the complexity grows as the number of training samples increases).

The Microsoft Cognitive Toolkit was originally launched in beta back in October 2016. It provides a set of robust intelligent services that enable developers to draw on AI within their own products. The revised version launched this week includes "hundreds" of new features that combine to make a simpler and more powerful toolkit. Microsoft originally built the Cognitive Toolkit to aid the development of its own internal services. It was devised as a method of accelerating the previously costly training procedures required to launch new neural networks.

Last week I had the opportunity to visit my graduate school alma mater, The University of Arizona where I studied artificial intelligence and image processing many years ago. I remember signing up for my first semester classes and electing to challenge myself with Professor Neifeld's neural network class. It already had the reputation of being one of the toughest classes requiring students to understand both the mathematical theory and real-world application of neural networks to solve classification and other problems. Neural Networks Before Cloud Computing Of course back then there wasn't cloud computing or easy access to parallel computing methods or deep learning Python libraries. As students, we had to program the algorithms by hand starting with the mathematics of a single neuron, the iterations to loop through all the neurons in each layer, and the algorithms to implement the backpropagation learning algorithms.

The technology industry is plagued with buzzword bingo in support of the fashion driven nature of the technology beast. Often confusing and occasionally downright ridiculous, we're never going to prevent smart ass marketers, ably supported by their anal-yst surrogates from making stuff up. The least some of us can do is make clear what is under discussion without mindlessly parroting what others say or conflating one concept with another. The latest in this stream of marketing laden garbage is AI or Artificial Intelligence, smeared with ML or Machine Learning and DL or Deep Learning. Add a soupçon of'robotics' just to amp the volume to something people can'get' and you have the potential for an exotic mix that both captivates the sentient mind but can also plant fear.