Deep Learning
Forecasting Economics and Financial Time Series: ARIMA vs. LSTM
Siami-Namini, Sima, Namin, Akbar Siami
Forecasting time series data is an important subject in economics, business, and finance. Traditionally, there are several techniques to effectively forecast the next lag of time series data such as univariate Autoregressive (AR), univariate Moving Average (MA), Simple Exponential Smoothing (SES), and more notably Autoregressive Integrated Moving Average (ARIMA) with its many variations. In particular, ARIMA model has demonstrated its outperformance in precision and accuracy of predicting the next lags of time series. With the recent advancement in computational power of computers and more importantly developing more advanced machine learning algorithms and approaches such as deep learning, new algorithms are developed to forecast time series data. The research question investigated in this article is that whether and how the newly developed deep learning-based algorithms for forecasting time series data, such as "Long Short-Term Memory (LSTM)", are superior to the traditional algorithms. The empirical studies conducted and reported in this article show that deep learning-based algorithms such as LSTM outperform traditional-based algorithms such as ARIMA model. More specifically, the average reduction in error rates obtained by LSTM is between 84 - 87 percent when compared to ARIMA indicating the superiority of LSTM to ARIMA. Furthermore, it was noticed that the number of training times, known as "epoch" in deep learning, has no effect on the performance of the trained forecast model and it exhibits a truly random behavior.
Graph Partition Neural Networks for Semi-Supervised Classification
Liao, Renjie, Brockschmidt, Marc, Tarlow, Daniel, Gaunt, Alexander L., Urtasun, Raquel, Zemel, Richard
We present graph partition neural networks (GPNN), an extension of graph neural networks (GNNs) able to handle extremely large graphs. GPNNs alternate between locally propagating information between nodes in small subgraphs and globally propagating information between the subgraphs. To efficiently partition graphs, we experiment with several partitioning algorithms and also propose a novel variant for fast processing of large scale graphs. We extensively test our model on a variety of semi-supervised node classification tasks. Experimental results indicate that GPNNs are either superior or comparable to state-of-the-art methods on a wide variety of datasets for graph-based semi-supervised classification. We also show that GPNNs can achieve similar performance as standard GNNs with fewer propagation steps.
Vulnerability of Deep Learning
The Renormalisation Group (RG) provides a framework in which it is possible to assess whether a deep-learning network is sensitive to small changes in the input data and hence prone to error, or susceptible to adversarial attack. Distinct classification outputs are associated with different RG fixed points and sensitivity to small changes in the input data is due to the presence of relevant operators at a fixed point. A numerical scheme, based on Monte Carlo RG ideas, is proposed for identifying the existence of relevant operators and the corresponding directions of greatest sensitivity in the input data. Thus, a trained deep-learning network may be tested for its robustness and, if it is vulnerable to attack, dangerous perturbations of the input data identified.
ARMDN: Associative and Recurrent Mixture Density Networks for eRetail Demand Forecasting
Mukherjee, Srayanta, Shankar, Devashish, Ghosh, Atin, Tathawadekar, Nilam, Kompalli, Pramod, Sarawagi, Sunita, Chaudhury, Krishnendu
Accurate demand forecasts can help on-line retail organizations better plan their supply-chain processes. The challenge, however, is the large number of associative factors that result in large, non-stationary shifts in demand, which traditional time series and regression approaches fail to model. In this paper, we propose a Neural Network architecture called AR-MDN, that simultaneously models associative factors, time-series trends and the variance in the demand. We first identify several causal features and use a combination of feature embeddings, MLP and LSTM to represent them. We then model the output density as a learned mixture of Gaussian distributions. The AR-MDN can be trained end-to-end without the need for additional supervision. We experiment on a dataset of an year's worth of data over tens-of-thousands of products from Flipkart. The proposed architecture yields a significant improvement in forecasting accuracy when compared with existing alternatives.
Folded Recurrent Neural Networks for Future Video Prediction
Oliu, Marc, Selva, Javier, Escalera, Sergio
Future video prediction is an ill-posed Computer Vision problem that recently received much attention. Its main challenges are the high variability in video content, the propagation of errors through time, and the non-specificity of the future frames: given a sequence of past frames there is a continuous distribution of possible futures. This work introduces bijective Gated Recurrent Units, a double mapping between the input and output of a GRU layer. This allows for recurrent auto-encoders with state sharing between encoder and decoder, stratifying the sequence representation and helping to prevent capacity problems. We show how with this topology only the encoder or decoder needs to be applied for input encoding and prediction, respectively. This reduces the computational cost and avoids re-encoding the predictions when generating a sequence of frames, mitigating the propagation of errors. Furthermore, it is possible to remove layers from an already trained model, giving an insight to the role performed by each layer and making the model more explainable. We evaluate our approach on three video datasets, outperforming state of the art prediction results on MMNIST and UCF101, and obtaining competitive results on KTH with 2 and 3 times less memory usage and computational cost than the best scored approach.
What are the prerequisites for a large-scale AI initiative? - Data Points
Over the last few months, I've had the chance to engage with customers and industry analysts about a range of topics in the field of Artificial Intelligence, and I've been struck by how effective the Sentient Enterprise is in addressing the most common questions and misconceptions about AI. Examining customer case studies is one of the best way to share knowledge and insights around how enterprises are driving business outcomes from AI technology. Case studies are practical, relatable and authentic; and we are fortunate to have some great reference accounts that allow us to publicly share their AI and deep learning success stories. For context, most of our AI case studies start with Rapid Analytic Consulting Engagements (RACE) based on an agile and experimental process to find and test new insights and produce results in weeks, not months. So, the starting point for telling these stories is identifying the business outcome we want to achieve, and then jumping into a range of deep neural net taxonomies, augmenting current platforms with requisite software and GPU enablers, and measuring the final results.
Let's evolve a neural network with a genetic algorithm--code included
Building the perfect deep learning network involves a hefty amount of art to accompany sound science. One way to go about finding the right hyperparameters is through brute force trial and error: Try every combination of sensible parameters, send them to your Spark cluster, go about your daily jive, and come back when you have an answer. But there's gotta be a better way! Here, we try to improve upon the brute force method by applying a genetic algorithm to evolve a network with the goal of achieving optimal hyperparameters in a fraction the time of a brute force search. Let's say it takes five minutes to train and evaluate a network on your dataset.
Demystifying Docker for Data Scientists โ A Docker Tutorial for Your Deep Learning Projects
The docker run command first creates a writeable container layer over the specified image, and then starts it using the specified command. In this sense containers can be called instances of an image. If the image is not found on the host machine, it will download it from Docker hub before executing the command. We get a new container every time docker run command is executed allowing us to have multiple instances of the same image. Normally if we run a container without options it will start and stop immediately.
How I implemented iPhone X's FaceID using Deep Learning in Python.
One of the most discussed features of the new iPhone X is the new unlocking method, the successor of TouchID: FaceID. Having created a bezel-less phone, Apple had to develop a new method to unlock the phone in a easy and fast way. While some competitors continued using a fingerprint sensor, placed in a different position, Apple decided to innovate and revolutionize the way we unlock a phone: by simply looking at it. Thanks to an advanced (and remarkably small) front facing depth-camera, iPhone X in able to create a 3D map of the face of the user. In addition, a picture of the user's face is captured using an infrared camera, that is more robust to changes in light and color of the environment. Using deep learning, the smartphone is able to learn the user face in great detail, thus recognizing him\her every time the phone is picked up by its owner.
How to train and deploy deep learning at scale
In five lines, you can describe how your architecture looks and then you can also specify what algorithms you want to use for training. There are a lot of other systems challenges associated with actually going end to end, from data to a deployed model. The existing software solutions don't really tackle a big set of these challenges. For example, regardless of the software you're using, it takes days to weeks to train a deep learning model. There's real open challenges of how to best use parallel and distributed computing both to train a particular model and in the context of tuning hyperparameters of different models. We also found out the vast majority of organizations that we've spoken to in the last year or so who are using deep learning for what I'd call mission-critical problems, are actually doing it with on-premise hardware.