Statistical Learning
A simple experiment in Machine Learning Studio
If you've never used Azure Machine Learning Studio before, this tutorial is for you. In this tutorial, we'll walk through how to use Studio for the first time to create a machine learning experiment. The experiment will test an analytical model that predicts the price of an automobile based on different variables such as make and technical specifications. This tutorial shows you the basics of how to drag-and-drop modules onto your experiment, connect them together, run the experiment, and look at the results. We're not going to discuss the general topic of machine learning or how to select and use the 100 built-in algorithms and data manipulation modules included in Studio.
Explaining variability in logistic regression
Hi All, I have built a logistic regression but I am not able to figure out which goodness of fit measure will help me know'How much variability of the dependent variable is being explained by the current model' I have calculated few pseudo r2 to measure this but as we know pseudo r2 is not a good way of measuring.
Which machine learning algorithm should I use?
The machine learning algorithm cheat sheet helps you to choose from a variety of machine learning algorithms to find the appropriate algorithm for your specific problems. A support vector machine (SVM) training algorithm finds the classifier represented by the normal vector \(w\) and bias \(b\) of the hyperplane. A support vector machine (SVM) training algorithm finds the classifier represented by the normal vector and bias of the hyperplane. Support vector machines (SVM) and other simpler models, which can be easily trained by solving convex optimization problems, gradually replaced neural networks in machine learning.
Nice Generalization of the K-NN Clustering Algorithm -- Also Useful for Data Reduction
You don't need to know K-NN to understand this article -- but click here if you want to learn more about it. You don't need a background in statistical science either. Let's describe this new algorithm and its various components, in simple English We are dealing here with a supervised learning problem, and more specifically, clustering (also called supervised classification.). In particular, we want to assign a class label to a new observation that does not belong to the training set. Instead of checking out individual points (the nearest neighbors) and using a majority (voting) rule to assign the new observation to a cluster based on nearest neighbor counts, we are checking out cliques of points, and focus on the nearest cliques rather than on the nearest points.
Efficient training-image based geostatistical simulation and inversion using a spatial generative adversarial neural network
Laloy, Eric, Hรฉrault, Romain, Jacques, Diederik, Linde, Niklas
Probabilistic inversion within a multiple-point statistics framework is still computationally prohibitive for large-scale problems. To partly address this, we introduce and evaluate a new training-image based simulation and inversion approach for complex geologic media. Our approach relies on a deep neural network of the spatial generative adversarial network (SGAN) type. After training using a training image (TI), our proposed SGAN can quickly generate 2D and 3D unconditional realizations. A key feature of our SGAN is that it defines a (very) low-dimensional parameterization, thereby allowing for efficient probabilistic (or deterministic) inversion using state-of-the-art Markov chain Monte Carlo (MCMC) methods. A series of 2D and 3D categorical TIs is first used to analyze the performance of our SGAN for unconditional simulation. The speed at which realizations are generated makes it especially useful for simulating over large grids and/or from a complex multi-categorical TI. Subsequently, synthetic inversion case studies involving 2D steady-state flow and 3D transient hydraulic tomography are used to illustrate the effectiveness of our proposed SGAN-based probabilistic inversion. For the 2D case, the inversion rapidly explores the posterior model distribution. For the 3D case, the inversion recovers model realizations that fit the data close to the target level and visually resemble the true model well. Future work will focus on the inclusion of direct conditioning data and application to continuous TIs.
An Ensemble Quadratic Echo State Network for Nonlinear Spatio-Temporal Forecasting
McDermott, Patrick L., Wikle, Christopher K.
Spatio-temporal data and processes are prevalent across a wide variety of scientific disciplines. These processes are often characterized by nonlinear time dynamics that include interactions across multiple scales of spatial and temporal variability. The data sets associated with many of these processes are increasing in size due to advances in automated data measurement, management, and numerical simulator output. Non- linear spatio-temporal models have only recently seen interest in statistics, but there are many classes of such models in the engineering and geophysical sciences. Tradi- tionally, these models are more heuristic than those that have been presented in the statistics literature, but are often intuitive and quite efficient computationally. We show here that with fairly simple, but important, enhancements, the echo state net- work (ESN) machine learning approach can be used to generate long-lead forecasts of nonlinear spatio-temporal processes, with reasonable uncertainty quantification, and at only a fraction of the computational expense of a traditional parametric nonlinear spatio-temporal models.
Adaptive Threshold Sampling and Estimation
Sampling is a fundamental problem in both computer science and statistics. A number of issues arise when designing a method based on sampling. These include statistical considerations such as constructing a good sampling design and ensuring there are good, tractable estimators for the quantities of interest as well as computational considerations such as designing fast algorithms for streaming data and ensuring the sample fits within memory constraints. Unfortunately, existing sampling methods are only able to address all of these issues in limited scenarios. We develop a framework that can be used to address these issues in a broad range of scenarios. In particular, it addresses the problem of drawing and using samples under some memory budget constraint. This problem can be challenging since the memory budget forces samples to be drawn non-independently and consequently, makes computation of resulting estimators difficult. At the core of the framework is the notion of a data adaptive thresholding scheme where the threshold effectively allows one to treat the non-independent sample as if it were drawn independently. We provide sufficient conditions for a thresholding scheme to allow this and provide ways to build and compose such schemes. Furthermore, we provide fast algorithms to efficiently sample under these thresholding schemes.
Optimal Alarms for Vehicular Collision Detection
Motro, Michael, Ghosh, Joydeep, Bhat, Chandra
Recent advances in in-vehicle awareness have end uses such as messages or warnings to drivers, automated braking or control, or fully driverless vehicles. There are similarly many sensors and communication devices that can provide awareness, and many models of traffic motion or human action that add predictive power. As there are many possible approaches, a single unified framework for intelligent vehicle design seems unlikely in the near future. However, there are certain tasks that are important for a variety of intelligent vehicle applications and (relatively) independent of the individual sensors or models used. One such task is vehicular collision detection: given the current position and state of two or more vehicles and a predictive model for their future motion, determine whether there is a significant chance of collision between vehicles in the near future. This task may sound trivial and is indeed simpler than the problems of scene reconstruction, predictive modeling or path planning. This simplicity allows vehicular collision detection to be framed as a self-contained task, with solutions that compromise between speed and robustness. Collision detection closely matches the theoretical problem of optimal alarm design [1], [2]. Optimal alarms were initially studied in the context of detecting bankruptcies or machine part failures [3] - critical events that should be detected in advance with high probability, much like collisions.
Weighted parallel SGD for distributed unbalanced-workload training system
Daning, Cheng, Shigang, Li, Yunquan, Zhang
Stochastic gradient descent (SGD) is a popular stochastic optimization method in machine learning. Traditional parallel SGD algorithms, e.g., SimuParallel SGD [1], often require all nodes to have the same performance or to consume equal quantities of data. However, these requirements are difficult to satisfy when the parallel SGD algorithms run in a heterogeneous computing environment; low-performance nodes will exert a negative influence on the final result. In this paper, we propose an algorithm called weighted parallel SGD (WP-SGD). WP-SGD combines weighted model parameters from different nodes in the system to produce the final output. WP-SGD makes use of the reduction in standard deviation to compensate for the loss from the inconsistency in performance of nodes in the cluster, which means that WP-SGD does not require that all nodes consume equal quantities of data. We also analyze the theoretical feasibility of running two other parallel SGD algorithms combined with WP-SGD in a heterogeneous environment. The experimental results show that WP-SGD significantly outperforms the traditional parallel SGD algorithms on distributed training systems with an unbalanced workload. TEX Templates March 18, 2018 1. Introduction The training process in machine learning can essentially be treated as the solving of the stochastic optimization problem.