Representation of time-varying signals in terms of a sparse set of atoms from an overcomplete dictionary is important in machine learning, and it has been proposed as one of the fundamental computations of the nervous system. Reconstruction algorithms create an estimate of the observed signal using the appropriate weighted atoms of an overcomplete dictionary. The overcompleteness of the dictionary creates a situation where there are multiple sets of dictionary atoms that reconstruct the signal equally well. Under these conditions, a solution that minimizes the number of dictionary atoms that have a nonzero contribution to the signal is preferred. Directly finding the solution that minimizes the number of atoms used from the dictionary or L0 norm is an NPcomplete problem, making it intractable even for moderately sized dictionaries. However, it has been shown that under very general conditions the solution that minimizes the sum of the absolute values of the contributions of the dictionary atoms or L1 norm can be used to identify the sparsest solution[Don06]. Although there is no analytical expression for the minimal L1 norm solution, there are effective algorithms for finding L1 minimal solutions, which has triggered an explosion of interest in the use of the L1 norm for sparse reconstructions.

We present a theoretically grounded approach to train deep neural networks, including recurrent networks, subject to class-dependent label noise. We propose two procedures for loss correction that are agnostic to both application domain and network architecture. They simply amount to at most a matrix inversion and multiplication, provided that we know the probability of each class being corrupted into another. We further show how one can estimate these probabilities, adapting a recent technique for noise estimation to the multi-class setting, and thus providing an end-to-end framework. Extensive experiments on MNIST, IMDB, CIFAR-10, CIFAR-100 and a large scale dataset of clothing images employing a diversity of architectures --- stacking dense, convolutional, pooling, dropout, batch normalization, word embedding, LSTM and residual layers --- demonstrate the noise robustness of our proposals. Incidentally, we also prove that, when ReLU is the only non-linearity, the loss curvature is immune to class-dependent label noise.

Big Data is one of the major challenges of statistical science and has numerous consequences from algorithmic and theoretical viewpoints. Big Data always involve massive data but they also often include online data and data heterogeneity. Recently some statistical methods have been adapted to process Big Data, like linear regression models, clustering methods and bootstrapping schemes. Based on decision trees combined with aggregation and bootstrap ideas, random forests were introduced by Breiman in 2001. They are a powerful nonparametric statistical method allowing to consider in a single and versatile framework regression problems, as well as two-class and multi-class classification problems. Focusing on classification problems, this paper proposes a selective review of available proposals that deal with scaling random forests to Big Data problems. These proposals rely on parallel environments or on online adaptations of random forests. We also describe how related quantities -- such as out-of-bag error and variable importance -- are addressed in these methods. Then, we formulate various remarks for random forests in the Big Data context. Finally, we experiment five variants on two massive datasets (15 and 120 millions of observations), a simulated one as well as real world data. One variant relies on subsampling while three others are related to parallel implementations of random forests and involve either various adaptations of bootstrap to Big Data or to "divide-and-conquer" approaches. The fifth variant relates on online learning of random forests. These numerical experiments lead to highlight the relative performance of the different variants, as well as some of their limitations.

Predicting business process behaviour is an important aspect of business process management. Motivated by research in natural language processing, this paper describes an application of deep learning with recurrent neural networks to the problem of predicting the next event in a business process. This is both a novel method in process prediction, which has largely relied on explicit process models, and also a novel application of deep learning methods. The approach is evaluated on two real datasets and our results surpass the state-of-the-art in prediction precision.

Machine learning techniques are being increasingly used as flexible non-linear fitting and prediction tools in the physical sciences. Fitting functions that exhibit multiple solutions as local minima can be analysed in terms of the corresponding machine learning landscape. Methods to explore and visualise molecular potential energy landscapes can be applied to these machine learning landscapes to gain new insight into the solution space involved in training and the nature of the corresponding predictions. In particular, we can define quantities analogous to molecular structure, thermodynamics, and kinetics, and relate these emergent properties to the structure of the underlying landscape. This Perspective aims to describe these analogies with examples from recent applications, and suggest avenues for new interdisciplinary research.

While modern day web applications aim to create impact at the civilization level, they have become vulnerable to adversarial activity, where the next cyber-attack can take any shape and can originate from anywhere. The increasing scale and sophistication of attacks, has prompted the need for a data driven solution, with machine learning forming the core of many cybersecurity systems. Machine learning was not designed with security in mind, and the essential assumption of stationarity, requiring that the training and testing data follow similar distributions, is violated in an adversarial domain. In this paper, an adversary's view point of a classification based system, is presented. Based on a formal adversarial model, the Seed-Explore-Exploit framework is presented, for simulating the generation of data driven and reverse engineering attacks on classifiers. Experimental evaluation, on 10 real world datasets and using the Google Cloud Prediction Platform, demonstrates the innate vulnerability of classifiers and the ease with which evasion can be carried out, without any explicit information about the classifier type, the training data or the application domain. The proposed framework, algorithms and empirical evaluation, serve as a white hat analysis of the vulnerabilities, and aim to foster the development of secure machine learning frameworks.

Simple Temporal Networks (STNs) provide a powerful and general tool for representing conjunctions of maximum delay constraints over ordered pairs of temporal variables. In this paper we introduce Hyper Temporal Networks (HyTNs), a strict generalization of STNs, to overcome the limitation of considering only conjunctions of constraints but maintaining a practical efficiency in the consistency check of the instances. In a Hyper Temporal Network a single temporal hyperarc constraint may be defined as a set of two or more maximum delay constraints which is satisfied when at least one of these delay constraints is satisfied. HyTNs are meant as a light generalization of STNs offering an interesting compromise. On one side, there exist practical pseudo-polynomial time algorithms for checking consistency and computing feasible schedules for HyTNs. On the other side, HyTNs offer a more powerful model accommodating natural constraints that cannot be expressed by STNs like Trigger off exactly delta min before (after) the occurrence of the first (last) event in a set., which are used to represent synchronization events in some process aware information systems/workflow models proposed in the literature.

Competition amongst universities is set to increase with institutions closely differentiating themselves to attract and retain the best quality students, academics and staff. Key to this differentiation will be an extensive technology adoption and innovation strategy, enhancing the student experience, delivery of learning content, community engagement and campus management. The education technology (Edutech) market in Australia is expected to grow significantly amidst increasing student demand for education services and technology innovation, competition amongst institutions and decreasing acquisition costs. Frost & Sullivan anticipates that as the learning experience becomes increasingly digitised, technologies and solutions incorporating big data and analytics, collaboration, Augmented / Virtual Reality technology, Artificial Intelligence and learning management systems will play a key role within universities in the coming years. Frost & Sullivan's most recent analysis, Australian Edutech Market: Key Trends, Technologies and Opportunities 2016-2022 finds that the Australian Edutech Market is expected to grow to AUD 1.7 Billion by 2022.

SAP SE (NYSE: SAP) today announced it was ranked as a leader in "The Forrester Wave: Predictive Analytics And Machine Learning Solutions, Q1 2017." According to Forrester, "SAP offers comprehensive data science tools to build models, but it is also the biggest enterprise application company on the planet. This puts SAP in a unique position to create tools that allow business users with no data science knowledge to use data-scientist-created models in applications. SAP's solution offers the data tools that enterprise data scientists expect, but it also offers distinguished automation tools to train models." A finding in the report is that the predictive analytics and machine learning (PAML) market is forecasted to experience a 15 percent compound annual growth rate (CAGR) through 2021.*

Please email your data science related quandaries to rachel@fast.ai. Note that questions are edited for clarity and brevity. In the last week I received two questions with diametrically opposed premises: one was excited that machine learning is now automated, the other was concerned that machine learning takes too many years of study. Q1: I heard that Google Cloud announced that entrepreneurs can easily and quickly build on top of ML/NLP APIs. Is this statement true: "The future of ML and data post Google Cloud - the future is here, NLP and speech advancements have been figured out by Google and are accessible by API. The secret sauce has been commoditized so you can build your secret sauce on top of it. The time to secret sauce is getting shorter to shorter"?