Deep Learning
Report: State of Artificial Intelligence in India - 2020
Artificial Intelligence or AI is a field of Data Science that trains computers to learn from experience, adjust to inputs, and perform tasks of certain cognitive levels. Over the last few years, AI has emerged as a significant data science function and, by utilizing advanced algorithms and computing power, AI is transforming the functional, operational, and strategic landscape of various business domains. AI algorithms are designed to make decisions, often using real-time data. Using sensors, digital data, and even remote inputs, AI algorithms combine information from a variety of different sources, analyze the data instantly, and act on the insights derived from the data. Most AI technologies – from advanced recommendation engines to self-driving cars – rely on diverse deep learning models. By utilizing these complex models, AI professionals are able to train computers to accomplish specific tasks by recognizing patterns in the data. Analytics India Magazine (AIM), in association with Jigsaw Academy, has developed this study on the Artificial Intelligence market to understand the developments of the AI market in India, covering the market in terms of Industry and Company Type. Moreover, the study delves into the market size of the different categories of AI and Analytics startups / boutique firms. As a part of the broad Data Science domain, the Artificial Intelligence technology function has so far been classified as an emerging technology segment. Moreover, the AI market in India has, till now, been dominated by the MNC Technology and the GIC or Captive firms. Domestic firms, Indian startups, and even International Technology startups across various sectors have, so far, not made a significant investment, in terms of operations and scale, in the Indian AI market. Additionally, IT services and Boutique AI & Analytics firms had not, till a couple of years ago, developed full-fledged AI offerings in India for their clients.
Artificial Intelligence Can Create Sound Tracks for Silent Videos
Researchers Ghose and Prevost created a deep learning algorithm which, given a silent video, can generate a realistic sounding synchronized soundtrack. Frequently, movies had added sound effects which were not recorded during after the recording to make it feel more realistic in a process called "Foley". Researchers at the university of Texas turned to deep learning to automate this process. They trained a neural network on 12 popular movie events where directors frequently add Foley effects. Their neural network classifies the class of the sound to generate, and also has a sequential network that generates the sound.
Bayesian deep learning for mapping via auxiliary information: a new era for geostatistics?
Kirkwood, Charlie, Economou, Theo, Pugeault, Nicolas
For geospatial modelling and mapping tasks, variants of kriging - the spatial interpolation technique developed by South African mining engineer Danie Krige - have long been regarded as the established geostatistical methods. However, kriging and its variants (such as regression kriging, in which auxiliary variables or derivatives of these are included as covariates) are relatively restrictive models and lack capabilities that have been afforded to us in the last decade by deep neural networks. Principal among these is feature learning - the ability to learn filters to recognise task-specific patterns in gridded data such as images. Here we demonstrate the power of feature learning in a geostatistical context, by showing how deep neural networks can automatically learn the complex relationships between point-sampled target variables and gridded auxiliary variables (such as those provided by remote sensing), and in doing so produce detailed maps of chosen target variables. At the same time, in order to cater for the needs of decision makers who require well-calibrated probabilities, we obtain uncertainty estimates via a Bayesian approximation known as Monte Carlo dropout. In our example, we produce a national-scale probabilistic geochemical map from point-sampled assay data, with auxiliary information provided by a terrain elevation grid. Unlike traditional geostatistical approaches, auxiliary variable grids are fed into our deep neural network raw. There is no need to provide terrain derivatives (e.g. slope angles, roughness, etc) because the deep neural network is capable of learning these and arbitrarily more complex derivatives as necessary to maximise predictive performance. We hope our results will raise awareness of the suitability of Bayesian deep learning - and its feature learning capabilities - for large-scale geostatistical applications where uncertainty matters.
Neural Time-Dependent Partial Differential Equation
Hu, Yihao, Zhao, Tong, Xu, Zhiliang, Lin, Lizhen
Partial differential equations(PDEs) play a crucial role in studying a vast number of problems in science and engineering. Numerically solving nonlinear and/or high-dimensional PDEs is often a challenging task. Inspired by the traditional finite difference and finite element methods and emerging advancements in machine learning, we propose a sequence deep learning framework called Neural-PDE, which allows to automatically learn governing rules of any time-dependent PDE system from existing data by using a bidirectional LSTM encoder, and predict the next n time steps data. One critical feature of our proposed framework is that the Neural-PDE is able to simultaneously learn and simulate the multiscale variables. We test the Neural-PDE by a range of examples from one-dimensional PDEs to a high-dimensional and nonlinear complex fluids model. The results show that the Neural-PDE is capable of learning the initial conditions, boundary conditions and differential operators without the knowledge of the specific form of a PDE system. In our experiments the Neural-PDE can efficiently extract the dynamics within 20 epochs training, and produces accurate predictions. Furthermore, unlike the traditional machine learning approaches in learning PDE such as CNN and MLP which require vast parameters for model precision, Neural-PDE shares parameters across all time steps, thus considerably reduces the computational complexity and leads to a fast learning algorithm.
Discovering Generative Models from Event Logs: Data-driven Simulation vs Deep Learning
Camargo, Manuel, Dumas, Marlon, Gonzalez-Rojas, Oscar
A generative model is a statistical model that is able to generate new data instances from previously observed ones. In the context of business processes, a generative model creates new execution traces from a set of historical traces, also known as an event log. Two families of generative process simulation models have been developed in previous work: data-driven simulation models and deep learning models. Until now, these two approaches have evolved independently and their relative performance has not been studied. This paper fills this gap by empirically comparing a data-driven simulation technique with multiple deep learning techniques, which construct models are capable of generating execution traces with timestamped events. The study sheds light into the relative strengths of both approaches and raises the prospect of developing hybrid approaches that combine these strengths.
Enhancing the Interpretability of Deep Models in Heathcare Through Attention: Application to Glucose Forecasting for Diabetic People
De Bois, Maxime, Yacoubi, Mounîm A. El, Ammi, Mehdi
The adoption of deep learning in healthcare is hindered by their "black box" nature. In this paper, we explore the RETAIN architecture for the task of glusose forecasting for diabetic people. By using a two-level attention mechanism, the recurrent-neural-network-based RETAIN model is interpretable. We evaluate the RETAIN model on the type-2 IDIAB and the type-1 OhioT1DM datasets by comparing its statistical and clinical performances against two deep models and three models based on decision trees. We show that the RETAIN model offers a very good compromise between accuracy and interpretability, being almost as accurate as the LSTM and FCN models while remaining interpretable. We show the usefulness of its interpretable nature by analyzing the contribution of each variable to the final prediction. It revealed that signal values older than one hour are not used by the RETAIN model for the 30-minutes ahead of time prediction of glucose. Also, we show how the RETAIN model changes its behavior upon the arrival of an event such as carbohydrate intakes or insulin infusions. In particular, it showed that the patient's state before the event is particularily important for the prediction. Overall the RETAIN model, thanks to its interpretability, seems to be a very promissing model for regression or classification tasks in healthcare.
Artificial Intelligence versus Maya Angelou: Experimental evidence that people cannot differentiate AI-generated from human-written poetry
The release of openly available, robust natural language generation algorithms (NLG) has spurred much public attention and debate. One reason lies in the algorithms' purported ability to generate human-like text across various domains. Empirical evidence using incentivized tasks to assess whether people (a) can distinguish and (b) prefer algorithm-generated versus human-written text is lacking. We conducted two experiments assessing behavioral reactions to the state-of-the-art Natural Language Generation algorithm GPT-2 (Ntotal = 830). Using the identical starting lines of human poems, GPT-2 produced samples of poems. From these samples, either a random poem was chosen (Human-out-of-the-loop) or the best one was selected (Human-in-the-loop) and in turn matched with a human-written poem. In a new incentivized version of the Turing Test, participants failed to reliably detect the algorithmically-generated poems in the Human-in-the-loop treatment, yet succeeded in the Human-out-of-the-loop treatment. Further, people reveal a slight aversion to algorithm-generated poetry, independent on whether participants were informed about the algorithmic origin of the poem (Transparency) or not (Opacity). We discuss what these results convey about the performance of NLG algorithms to produce human-like text and propose methodologies to study such learning algorithms in human-agent experimental settings.
SNoRe: Scalable Unsupervised Learning of Symbolic Node Representations
Mežnar, Sebastian, Lavrač, Nada, Škrlj, Blaž
Learning from real-life complex networks is a lively research area, with recent advances in learning information-rich, low-dimensional network node representations. However, state-of-the-art methods offer little insights as the features that constitute the learned node representations are not interpretable and are as such less applicable to sensitive settings in biomedical or user profiling tasks, where bias detection is highly relevant. The proposed SNoRe (Symbolic Node Representations) algorithm is capable of learning symbolic, human-understandable representations of individual network nodes based on the similarity of neighborhood hashes to nodes chosen as features. SNoRe's interpretable features are suitable for direct explanation of individual predictions, which we demonstrate by coupling it with the widely used instance explanation tool SHAP to obtain nomograms representing the relevance of individual features for a given classification, which is to our knowledge one of the first such attempts in a structural node embedding setting. In the experimental evaluation on 11 real-life datasets, SNoRe proved to be competitive to strong baselines, such as variational graph autoencoders, node2vec and LINE. The vectorized implementation of SNoRe scales to large networks, making it suitable for many contemporary network analysis tasks.
Understanding and Exploiting Dependent Variables with Deep Metric Learning
Mahony, Niall O', Campbell, Sean, Carvalho, Anderson, Krpalkova, Lenka, Velasco-Hernandez, Gustavo, Riordan, Daniel, Walsh, Joseph
Deep Metric Learning (DML) approaches learn to represent inputs to a lower-dimensional latent space such that the distance between representations in this space corresponds with a predefined notion of similarity. This paper investigates how the mapping element of DML may be exploited in situations where the salient features in arbitrary classification problems vary over time or due to changing underlying variables. Examples of such variable features include seasonal and time-of-day variations in outdoor scenes in place recognition tasks for autonomous navigation and age/gender variations in human/animal subjects in classification tasks for medical/ethological studies. Through the use of visualisation tools for observing the distribution of DML representations per each query variable for which prior information is available, the influence of each variable on the classification task may be better understood. Based on these relationships, prior information on these salient background variables may be exploited at the inference stage of the DML approach by using a clustering algorithm to improve classification performance. This research proposes such a methodology establishing the saliency of query background variables and formulating clustering algorithms for better separating latent-space representations at run-time. The paper also discusses online management strategies to preserve the quality and diversity of data and the representation of each class in the gallery of embeddings in the DML approach. We also discuss latent works towards understanding the relevance of underlying/multiple variables with DML.
Perceptual Deep Neural Networks: Adversarial Robustness through Input Recreation
Vargas, Danilo Vasconcellos, Liao, Bingli, Kanzaki, Takahiro
Adversarial examples have shown that albeit highly accurate, models learned by machines, differently from humans,have many weaknesses. However, humans' perception is also fundamentally different from machines, because we do not see the signals which arrive at the retina but a rather complex recreation of them. In this paper, we explore how machines could recreate the input as well as investigate the benefits of such an augmented perception. In this regard, we propose Perceptual Deep Neural Networks ($\varphi$DNN) which also recreate their own input before further processing. The concept is formalized mathematically and two variations of it are developed (one based on inpainting the whole image and the other based on a noisy resized super resolution recreation). Experiments reveal that $\varphi$DNNs can reduce attacks' accuracy substantially, surpassing state-of-the-art defenses in 87% of the tests for adversarial training variations and 100% of the tests when only comparing with other pre-processing type of defenses. Moreover, the recreation process intentionally corrupts the input image. Interestingly, we show by ablation tests that corrupting the input is, although counter-intuitive,beneficial. This suggests that the blind-spot in vertebrates might also be, analogously, the precursor of visual robustness. Thus, $\varphi$DNNs reveal that input recreation has strong benefits for artificial neural networks similar to biological ones, shedding light into the importance of the blind-spot and starting an area of perception models for robust recognition in artificial intelligence.