Statistical Learning
A unified machine learning approach to time series forecasting applied to demand at emergency departments
Vollmer, Michaela A. C., Glampson, Ben, Mellan, Thomas A., Mishra, Swapnil, Mercuri, Luca, Costello, Ceire, Klaber, Robert, Cooke, Graham, Flaxman, Seth, Bhatt, Samir
There were 25.6 million attendances at Emergency Departments (EDs) in England in 2019 corresponding to an increase of 12 million attendances over the past ten years. The steadily rising demand at EDs creates a constant challenge to provide adequate quality of care while maintaining standards and productivity. Managing hospital demand effectively requires an adequate knowledge of the future rate of admission. Using 8 years of electronic admissions data from two major acute care hospitals in London, we develop a novel ensemble methodology that combines the outcomes of the best performing time series and machine learning approaches in order to make highly accurate forecasts of demand, 1, 3 and 7 days in the future. Both hospitals face an average daily demand of 208 and 106 attendances respectively and experience considerable volatility around this mean. However, our approach is able to predict attendances at these emergency departments one day in advance up to a mean absolute error of +/- 14 and +/- 10 patients corresponding to a mean absolute percentage error of 6.8% and 8.6% respectively. Our analysis compares machine learning algorithms to more traditional linear models. We find that linear models often outperform machine learning methods and that the quality of our predictions for any of the forecasting horizons of 1, 3 or 7 days are comparable as measured in MAE. In addition to comparing and combining state-of-the-art forecasting methods to predict hospital demand, we consider two different hyperparameter tuning methods, enabling a faster deployment of our models without compromising performance. We believe our framework can readily be used to forecast a wide range of policy relevant indicators.
Random extrapolation for primal-dual coordinate descent
Alacaoglu, Ahmet, Fercoq, Olivier, Cevher, Volkan
We introduce a randomly extrapolated primal-dual coordinate descent method that adapts to sparsity of the data matrix and the favorable structures of the objective function. Our method updates only a subset of primal and dual variables with sparse data, and it uses large step sizes with dense data, retaining the benefits of the specific methods designed for each case. In addition to adapting to sparsity, our method attains fast convergence guarantees in favorable cases \textit{without any modifications}. In particular, we prove linear convergence under metric subregularity, which applies to strongly convex-strongly concave problems and piecewise linear quadratic functions. We show almost sure convergence of the sequence and optimal sublinear convergence rates for the primal-dual gap and objective values, in the general convex-concave case. Numerical evidence demonstrates the state-of-the-art empirical performance of our method in sparse and dense settings, matching and improving the existing methods.
Minimum Relative Entropy Inference for Normal and Monte Carlo Distributions
Colasante, Marcello, Meucci, Attilio
Inference is ubiquitous in financial applications: stress-testing and scenario analysis, such as in[Mina and Xiao, 2001], explore the consequences of specific market scenarios on the distribution of the portfolio loss. Similar, portfolio construction techniques such as [Black and Litterman, 1990] inject views on specific factor returns into the estimated distribution of a broad market. A general approach to perform inference under partial information based on the principle of minimum relative entropy (MRE) was explored in [Meucci, 2010]. In the original paper, the general theory was supported by two applications: an analytical solution under normality, and a numerical algorithm for distributions represented by scenarios, such as Monte Carlo, historical, or categorical. Here we enhance both the analytical and the numerical implementations of [Meucci, 2010] drawing from results in [Colasante, 2019]. In Section 2 we state well-known results to set the notation and background.
Orthogonally Decoupled Variational Fourier Features
Azzimonti, Dario, Schรผrch, Manuel, Benavoli, Alessio, Zaffalon, Marco
Sparse inducing points have long been a standard method to fit Gaussian processes to big data. In the last few years, spectral methods that exploit approximations of the covariance kernel have shown to be competitive. In this work we exploit a recently introduced orthogonally decoupled variational basis to combine spectral methods and sparse inducing points methods. We show that the method is competitive with the state-of-the-art on synthetic and on real-world data.
Distributed Graph Convolutional Networks
Scardapane, Simone, Spinelli, Indro, Di Lorenzo, Paolo
The aim of this work is to develop a fully-distributed algorithmic framework for training graph convolutional networks (GCNs). The proposed method is able to exploit the meaningful relational structure of the input data, which are collected by a set of agents that communicate over a sparse network topology. After formulating the centralized GCN training problem, we first show how to make inference in a distributed scenario where the underlying data graph is split among different agents. Then, we propose a distributed gradient descent procedure to solve the GCN training problem. The resulting model distributes computation along three lines: during inference, during back-propagation, and during optimization. Convergence to stationary solutions of the GCN training problem is also established under mild conditions. Finally, we propose an optimization criterion to design the communication topology between agents in order to match with the graph describing data relationships. A wide set of numerical results validate our proposal. To the best of our knowledge, this is the first work combining graph convolutional neural networks with distributed optimization.
Deep Graph Contrastive Representation Learning
Zhu, Yanqiao, Xu, Yichen, Yu, Feng, Liu, Qiang, Wu, Shu, Wang, Liang
Graph representation learning nowadays becomes fundamental in analyzing graph-structured data. Inspired by recent success of contrastive methods, in this paper, we propose a novel framework for unsupervised graph representation learning by leveraging a contrastive objective at the node level. Specifically, we generate two graph views by corruption and learn node representations by maximizing the agreement of node representations in these two views. To provide diverse node contexts for the contrastive objective, we propose a hybrid scheme for generating graph views on both structure and attribute levels. Besides, we provide theoretical justification behind our motivation from two perspectives, mutual information and the classical triplet loss. We perform empirical experiments on both transductive and inductive learning tasks using a variety of real-world datasets. Experimental experiments demonstrate that despite its simplicity, our proposed method consistently outperforms existing state-of-the-art methods by large margins. Moreover, our unsupervised method even surpasses its supervised counterparts on transductive tasks, demonstrating its great potential in real-world applications.
Streaming Coresets for Symmetric Tensor Factorization
Chhaya, Rachit, Choudhari, Jayesh, Dasgupta, Anirban, Shit, Supratim
Factorizing tensors has recently become an important optimization module in a number of machine learning pipelines, especially in latent variable models. We show how to do this efficiently in the streaming setting. Given a set of $n$ vectors, each in $\mathbb{R}^d$, we present algorithms to select a sublinear number of these vectors as coreset, while guaranteeing that the CP decomposition of the $p$-moment tensor of the coreset approximates the corresponding decomposition of the $p$-moment tensor computed from the full data. We introduce two novel algorithmic techniques: online filtering and kernelization. Using these two, we present six algorithms that achieve different tradeoffs of coreset size, update time and working space, beating or matching various state of the art algorithms. In the case of matrices ($2$-ordered tensor), our online row sampling algorithm guarantees $(1 \pm \epsilon)$ relative error spectral approximation. We show applications of our algorithms in learning single topic modeling.
Writing your first hello world in machine learning with Python
Machine Learning is the process of taking data as input, identifying the trends and patterns in data and giving a program as output. This program also or model is the representation of the patterns that define the data. New data can be given as input to this model/program and it will be able to classify it or make predictions on it. Python has been found to be a simple and easy to learn language that works very well with requirements of machine learning. Python has been designed to favor data analysis.
9 Key Machine Learning Algorithms Explained in Plain English
Machine learning [https://gum.co/pGjwd] is changing the world. Google uses machine learning to suggest search results to users. Netflix uses it to recommend movies for you to watch. Facebook uses machine learning to suggest people you may know. Machine learning has never been more important. At the same time, understanding machine learning is hard. The field is full of jargon. And the number of different ML algorithms grows each year. This article will introduce you to the fundamental concepts