About this course: How do we infer which genes orchestrate various processes in the cell? How did humans migrate out of Africa and spread around the world? In this class, we will see that these two seemingly different questions can be addressed using similar algorithmic and machine learning techniques arising from the general problem of dividing data points into distinct clusters. In the first half of the course, we will introduce algorithms for clustering a group of objects into a collection of clusters based on their similarity, a classic problem in data science, and see how these algorithms can be applied to gene expression data. In the second half of the course, we will introduce another classic tool in data science called principal components analysis that can be used to preprocess multidimensional data before clustering in an effort to greatly reduce the number dimensions without losing much of the "signal" in the data.

About this course: Want to make sense of the volumes of data you have collected? Need to incorporate data-driven decisions into your process? This course provides an overview of machine learning techniques to explore, analyze, and leverage data. You will be introduced to tools and algorithms you can use to create machine learning models that learn from data, and to scale those models up to big data problems. At the end of the course, you will be able to: • Design an approach to leverage data using the steps in the machine learning process.

Scientists have solved the puzzle of the so-called "Frankenstein dinosaur", which seems to consist of body parts from unrelated species. A new study suggests that it is in fact the missing link between plant-eating dinosaurs, such as Stegosaurus, and carnivorous dinosaurs, like T. rex. The finding provides fresh insight on the evolution of the group of dinos known as the ornithischians. The study is published in the Royal Society journal Biology Letters. Matthew Baron, a PhD student at Cambridge University, told BBC News that his assessment indicated that the Frankenstein dinosaur was one of the very first ornithischians, a group that included familiar beasts such as the horned Triceratops, and Stegosaurus which sported an array of bony plates along its back.

Probabilistic inversion within a multiple-point statistics framework is still computationally prohibitive for large-scale problems. To partly address this, we introduce and evaluate a new training-image based simulation and inversion approach for complex geologic media. Our approach relies on a deep neural network of the spatial generative adversarial network (SGAN) type. After training using a training image (TI), our proposed SGAN can quickly generate 2D and 3D unconditional realizations. A key feature of our SGAN is that it defines a (very) low-dimensional parameterization, thereby allowing for efficient probabilistic (or deterministic) inversion using state-of-the-art Markov chain Monte Carlo (MCMC) methods. A series of 2D and 3D categorical TIs is first used to analyze the performance of our SGAN for unconditional simulation. The speed at which realizations are generated makes it especially useful for simulating over large grids and/or from a complex multi-categorical TI. Subsequently, synthetic inversion case studies involving 2D steady-state flow and 3D transient hydraulic tomography are used to illustrate the effectiveness of our proposed SGAN-based probabilistic inversion. For the 2D case, the inversion rapidly explores the posterior model distribution. For the 3D case, the inversion recovers model realizations that fit the data close to the target level and visually resemble the true model well. Future work will focus on the inclusion of direct conditioning data and application to continuous TIs.

During reactive transport modeling, the computational cost associated with chemical reaction calculations is often 10-100 times higher than that of transport calculations. Most of these costs results from chemical equilibrium calculations that are performed at least once in every mesh cell and at every time step of the simulation. Calculating chemical equilibrium is an iterative process, where each iteration is in general so computationally expensive that even if every calculation converged in a single iteration, the resulting speedup would not be significant. Thus, rather than proposing a fast-converging numerical method for solving chemical equilibrium equations, we present a machine learning method that enables new equilibrium states to be quickly and accurately estimated, whenever a previous equilibrium calculation with similar input conditions has been performed. We demonstrate the use of this smart chemical equilibrium method in a reactive transport modeling example and show that, even at early simulation times, the majority of all equilibrium calculations are quickly predicted and, after some time steps, the machine-learning-accelerated chemical solver has been fully trained to rapidly perform all subsequent equilibrium calculations, resulting in speedups of almost two orders of magnitude. We remark that our new on-demand machine learning method can be applied to any case in which a massive number of sequential/parallel evaluations of a computationally expensive function $f$ needs to be done, $y=f(x)$. We remark, that, in contrast to traditional machine learning algorithms, our on-demand training approach does not require a statistics-based training phase before the actual simulation of interest commences. The introduced on-demand training scheme requires, however, the first-order derivatives $\partial f/\partial x$ for later smart predictions.

Spatio-temporal data and processes are prevalent across a wide variety of scientific disciplines. These processes are often characterized by nonlinear time dynamics that include interactions across multiple scales of spatial and temporal variability. The data sets associated with many of these processes are increasing in size due to advances in automated data measurement, management, and numerical simulator output. Non- linear spatio-temporal models have only recently seen interest in statistics, but there are many classes of such models in the engineering and geophysical sciences. Tradi- tionally, these models are more heuristic than those that have been presented in the statistics literature, but are often intuitive and quite efficient computationally. We show here that with fairly simple, but important, enhancements, the echo state net- work (ESN) machine learning approach can be used to generate long-lead forecasts of nonlinear spatio-temporal processes, with reasonable uncertainty quantification, and at only a fraction of the computational expense of a traditional parametric nonlinear spatio-temporal models.

Sampling is a fundamental problem in both computer science and statistics. A number of issues arise when designing a method based on sampling. These include statistical considerations such as constructing a good sampling design and ensuring there are good, tractable estimators for the quantities of interest as well as computational considerations such as designing fast algorithms for streaming data and ensuring the sample fits within memory constraints. Unfortunately, existing sampling methods are only able to address all of these issues in limited scenarios. We develop a framework that can be used to address these issues in a broad range of scenarios. In particular, it addresses the problem of drawing and using samples under some memory budget constraint. This problem can be challenging since the memory budget forces samples to be drawn non-independently and consequently, makes computation of resulting estimators difficult. At the core of the framework is the notion of a data adaptive thresholding scheme where the threshold effectively allows one to treat the non-independent sample as if it were drawn independently. We provide sufficient conditions for a thresholding scheme to allow this and provide ways to build and compose such schemes. Furthermore, we provide fast algorithms to efficiently sample under these thresholding schemes.

Recent advances in in-vehicle awareness have end uses such as messages or warnings to drivers, automated braking or control, or fully driverless vehicles. There are similarly many sensors and communication devices that can provide awareness, and many models of traffic motion or human action that add predictive power. As there are many possible approaches, a single unified framework for intelligent vehicle design seems unlikely in the near future. However, there are certain tasks that are important for a variety of intelligent vehicle applications and (relatively) independent of the individual sensors or models used. One such task is vehicular collision detection: given the current position and state of two or more vehicles and a predictive model for their future motion, determine whether there is a significant chance of collision between vehicles in the near future. This task may sound trivial and is indeed simpler than the problems of scene reconstruction, predictive modeling or path planning. This simplicity allows vehicular collision detection to be framed as a self-contained task, with solutions that compromise between speed and robustness. Collision detection closely matches the theoretical problem of optimal alarm design [1], [2]. Optimal alarms were initially studied in the context of detecting bankruptcies or machine part failures [3] - critical events that should be detected in advance with high probability, much like collisions.

Stochastic gradient descent (SGD) is a popular stochastic optimization method in machine learning. Traditional parallel SGD algorithms, e.g., SimuParallel SGD [1], often require all nodes to have the same performance or to consume equal quantities of data. However, these requirements are difficult to satisfy when the parallel SGD algorithms run in a heterogeneous computing environment; low-performance nodes will exert a negative influence on the final result. In this paper, we propose an algorithm called weighted parallel SGD (WP-SGD). WP-SGD combines weighted model parameters from different nodes in the system to produce the final output. WP-SGD makes use of the reduction in standard deviation to compensate for the loss from the inconsistency in performance of nodes in the cluster, which means that WP-SGD does not require that all nodes consume equal quantities of data. We also analyze the theoretical feasibility of running two other parallel SGD algorithms combined with WP-SGD in a heterogeneous environment. The experimental results show that WP-SGD significantly outperforms the traditional parallel SGD algorithms on distributed training systems with an unbalanced workload. TEX Templates March 18, 2018 1. Introduction The training process in machine learning can essentially be treated as the solving of the stochastic optimization problem.

We present a novel regularization scheme for training deep neural networks. The parameters of neural networks are usually unconstrained and have a dynamic range dispersed over the real line. Our key idea is to control the expressive power of the network by dynamically quantizing the range and set of values that the parameters can take. We formulate this idea using a novel end-to-end approach that regularizes the traditional classification loss function. Our regularizer is inspired by the Minimum Description Length principle. For each layer of the network, our approach optimizes a translation and scaling factor along with integer-valued parameters. We empirically compare BitNet to an equivalent unregularized model on the MNIST and CIFAR-10 datasets. We show that BitNet converges faster to a superior quality solution. Additionally, the resulting model is significantly smaller in size due to the use of integer parameters instead of floats.