This paper develops a theory of clustering and coding which combines a geometric model with a probabilistic model in a principled way. The geometric model is a Riemannian manifold with a Riemannian metric, ${g}_{ij}({\bf x})$, which we interpret as a measure of dissimilarity. The probabilistic model consists of a stochastic process with an invariant probability measure which matches the density of the sample input data. The link between the two models is a potential function, $U({\bf x})$, and its gradient, $\nabla U({\bf x})$. We use the gradient to define the dissimilarity metric, which guarantees that our measure of dissimilarity will depend on the probability measure. Finally, we use the dissimilarity metric to define a coordinate system on the embedded Riemannian manifold, which gives us a low-dimensional encoding of our original data.

One such application is intrusion detection in computer systems, where data points are typically digital messages transmitted over a network, and messages that are detected as outliers are considered likely to carry a threat [13, 17]. Another application is obstacle detection in autonomous car driving [18]. The ability to detect outliers is also important in scientific applications, where points detected as such are intrinsically more interesting than inliers, and should therefore be given more attention [59, 28]. A number of techniques can be used for outlier detection [12, 21, 36, 41, 51]. In practice, it is not only important to be able to detect outliers and inliers with high accuracy, one would also like to be able to explain why a machine learning model considers a sample as inlier or outlier. An interpretable explanatory feedback can indeed be used by a human operator for appropriate decision making. The data point could either be considered as benign and possibly incorporated to the dataset, or appropriate action might be taken. The problem of outlier explanation is shown schematically in Figure 1.

We build a rigorous bridge between deep networks (DNs) and approximation theory via spline functions and operators. Our key result is that a large class of DNs can be written as a composition of max-affine spline operators (MASOs), which provide a powerful portal through which to view and analyze their inner workings. For instance, conditioned on the input signal, the output of a MASO DN can be written as a simple affine transformation of the input. This implies that a DN constructs a set of signal-dependent, class-specific templates against which the signal is compared via a simple inner product; we explore the links to the classical theory of optimal classification via matched filters and the effects of data memorization. Going further, we propose a simple penalty term that can be added to the cost function of any DN learning algorithm to force the templates to be orthogonal with each other; this leads to significantly improved classifi- cation performance and reduced overfitting with no change to the DN architecture. The spline partition of the input signal space that is implicitly induced by a MASO directly links DNs to the theory of vector quantization (VQ) and K-means clustering, which opens up new geometric avenue to study how DNs organize signals in a hierarchical fashion. To validate the utility of the VQ interpretation, we develop and validate a new distance metric for signals and images that quantifies the difference between their VQ encodings. (This paper is a significantly expanded version of a paper with the same title that will appear at ICML 2018.)

We investigate structured sparsity methods for variable selection in regression problems where the target depends nonlinearly on the inputs. We focus on general nonlinear functions not limiting a priori the function space to additive models. We propose two new regularizers based on partial derivatives as nonlinear equivalents of group lasso and elastic net. We formulate the problem within the framework of learning in reproducing kernel Hilbert spaces and show how the variational problem can be reformulated into a more practical finite dimensional equivalent. We develop a new algorithm derived from the ADMM principles that relies solely on closed forms of the proximal operators. We explore the empirical properties of our new algorithm for Nonlinear Variable Selection based on Derivatives (NVSD) on a set of experiments and confirm favourable properties of our structured-sparsity models and the algorithm in terms of both prediction and variable selection accuracy.

In recent years data sets have grown in size and dimension with the proliferation of advanced data acquisition techniques. We have been able to use such data meaningfully not only because the computation power has increased to match the size, but also due to the paradigm shift in data analysis techniques that handle such data. A prime example is Machine Learning (ML). As a result new applications and techniques are emerging more frequently than ever before. Examples include object classification with applications in medicine (e.g., brain image analysis) and security (e.g., face classification) [1]. In many such applications, items in a data set are considered as points in some feature space of the underlying data, enabling us to interpret the data set as a "point cloud" in a suitably identified space. Even though, in certain cases the feature space is easily identifiable, in many other cases identifying a feature space could be a less obvious task.

This paper presents a practical approach for detecting non-stationarity in time series prediction. This method is called SAFE and works by monitoring the evolution of the spectral contents of time series through a distance function. This method is designed to work in combination with state-of-the-art machine learning methods in real time by informing the online predictors to perform necessary adaptation when a non-stationarity presents. We also propose an algorithm to proportionally include some past data in the adaption process to overcome the Catastrophic Forgetting problem. To validate our hypothesis and test the effectiveness of our approach, we present comprehensive experiments in different elements of the approach involving artificial and real-world datasets. The experiments show that the proposed method is able to significantly save computational resources in term of processor or GPU cycles while maintaining high prediction performances.

Forecasting is a core part of time series analysis as it tries tries to predict the value of the analysed signal. Forecasting is one of the hardest problems in predictive analytics because it's not always obvious what attributes can explain the future values of the signal and because you often will have less data than you would like to have, for example, if you have monthly data over a 4 year period you will basically have 48 data points. As time series analysis data is temporal, you will often have one data points per timestamp. The motto, "the more data the better," is true only up to a certain point, particularly when running a time series analysis. Adding more data can actually negatively impact your model.

Many businesses are beginning to rely on large scale data analytics for greater insights into their customers' behavior and their business requirements. Simplifying the process so that a wider range of employees can make conclusions from the massive amounts of data is important and can lead to more profits and better customer service. Harp-DAAL is a framework developed at Indiana University that brings together the capabilities of big data (Hadoop) and techniques that have previously been adopted for high performance computing. Together, employees can become more productive and gain deeper insights to massive amounts of data. Modern analytics systems are clusters of independent systems which need to be synchronized in order to make sense of all of the data.

Description This easy video tutorial is the fastest way to master modern data science best practices and use them to promote timely, evidence-based decision-making! Applied Data Mining LiveLessons demystifies current best practices, showing how to uncover hidden patterns and leverage them to improve all aspects of business performance. Drawing on extensive experience as a researcher, practitioner, and instructor, Dr. Dursun Delen shows you exactly how analytics and data mining work, why they've become so important, and how to apply them to your problems. Delen reviews key concepts, applications, and challenges; introduces advanced tools and technologies, including IBM Watson; and discusses privacy concerns associated with modern data mining. You'll watch him demonstrate prediction, classification, decision trees, and cluster analysis...key algorithms such as nearest neighbor...artificial neural networks...regression and time-series forecasting...text analytics and sentiment analysis...big data techniques, technologies, and more.

Analysis of reactive-diffusion simulations requires a large number of independent model runs. For each high-fidelity simulation, inputs are varied and the predicted mixing behavior is represented by changes in species concentration. It is then required to discern how the model inputs impact the mixing process. This task is challenging and typically involves interpretation of large model outputs. However, the task can be automated and substantially simplified by applying Machine Learning (ML) methods. In this paper, we present an application of an unsupervised ML method (called NTFk) using Non-negative Tensor Factorization (NTF) coupled with a custom clustering procedure based on k-means to reveal hidden features in product concentration. An attractive aspect of the proposed ML method is that it ensures the extracted features are non-negative, which are important to obtain a meaningful deconstruction of the mixing processes. The ML method is applied to a large set of high-resolution FEM simulations representing reaction-diffusion processes in perturbed vortex-based velocity fields. The applied FEM ensures that species concentration are always non-negative. The simulated reaction is a fast irreversible bimolecular reaction. The reactive-diffusion model input parameters that control mixing include properties of velocity field, anisotropic dispersion, and molecular diffusion. We demonstrate the applicability of the ML method to produce a meaningful deconstruction of model outputs to discriminate between different physical processes impacting the reactants, their mixing, and the spatial distribution of the product. The presented ML analysis allowed us to identify additive features that characterize mixing behavior.