Deep Learning
Recurrent Deep Divergence-based Clustering for simultaneous feature learning and clustering of variable length time series
Trosten, Daniel J., Strauman, Andreas S., Kampffmeyer, Michael, Jenssen, Robert
The task of clustering unlabeled time series and sequences entails a particular set of challenges, namely to adequately model temporal relations and variable sequence lengths. If these challenges are not properly handled, the resulting clusters might be of suboptimal quality. As a key solution, we present a joint clustering and feature learning framework for time series based on deep learning. For a given set of time series, we train a recurrent network to represent, or embed, each time series in a vector space such that a divergence-based clustering loss function can discover the underlying cluster structure in an end-to-end manner. Unlike previous approaches, our model inherently handles multivariate time series of variable lengths and does not require specification of a distance-measure in the input space. On a diverse set of benchmark datasets we illustrate that our proposed Recurrent Deep Divergence-based Clustering approach outperforms, or performs comparable to, previous approaches.
Time Aggregation and Model Interpretation for Deep Multivariate Longitudinal Patient Outcome Forecasting Systems in Chronic Ambulatory Care
Norgeot, Beau, Lituiev, Dmytro, Glicksberg, Benjamin S., Butte, Atul J.
Clinical data for ambulatory care, which accounts for 90% of the nations healthcare spending, is characterized by relatively small sample sizes of longitudinal data, unequal spacing between visits for each patient, with unequal numbers of data points collected across patients. While deep learning has become state-of-the-art for sequence modeling, it is unknown which methods of time aggregation may be best suited for these challenging temporal use cases. Additionally, deep models are often considered uninterpretable by physicians which may prevent the clinical adoption, even of well performing models. We show that time-distributed-dense layers combined with GRUs produce the most generalizable models. Furthermore, we provide a framework for the clinical interpretation of the models.
Analyzing Federated Learning through an Adversarial Lens
Bhagoji, Arjun Nitin, Chakraborty, Supriyo, Mittal, Prateek, Calo, Seraphin
Federated learning distributes model training among a multitude of agents, who, guided by privacy concerns, perform training using their local data but share only model parameter updates, for iterative aggregation at the server. In this work, we explore the threat of model poisoning attacks on federated learning initiated by a single, non-colluding malicious agent where the adversarial objective is to cause the model to misclassify a set of chosen inputs with high confidence. We explore a number of strategies to carry out this attack, starting with simple boosting of the malicious agent's update to overcome the effects of other agents' updates. To increase attack stealth, we propose an alternating minimization strategy, which alternately optimizes for the training loss and the adversarial objective. We follow up by using parameter estimation for the benign agents' updates to improve on attack success. Finally, we use a suite of interpretability techniques to generate visual explanations of model decisions for both benign and malicious models and show that the explanations are nearly visually indistinguishable. Our results indicate that even a highly constrained adversary can carry out model poisoning attacks while simultaneously maintaining stealth, thus highlighting the vulnerability of the federated learning setting and the need to develop effective defense strategies.
Touchdown: Natural Language Navigation and Spatial Reasoning in Visual Street Environments
Chen, Howard, Suhr, Alane, Misra, Dipendra, Snavely, Noah, Artzi, Yoav
We study the problem of jointly reasoning about language and vision through a navigation and spatial reasoning task. We introduce the Touchdown task and dataset, where an agent must first follow navigation instructions in a real-life visual urban environment to a goal position, and then identify in the observed image a location described in natural language to find a hidden object. The data contains 9,326 examples of English instructions and spatial descriptions paired with demonstrations. We perform qualitative linguistic analysis, and show that the data displays richer use of spatial reasoning compared to related resources. Empirical analysis shows the data presents an open challenge to existing methods.
Machine Learning on Electronic Health Records: Models and Features Usages to predict Medication Non-Adherence
Janssoone, Thomas, Bic, Clรฉmence, Kanoun, Dorra, Hornus, Pierre, Rinder, Pierre
Adherence can be defined as "the extent to which patients take their medications as prescribed by their healthcare providers"[Osterberg and Blaschke, 2005]. World Health Organization's reports point out that, in developed countries, only about 50% of patients with chronic diseases correctly follow their treatments. This severely compromises the efficiency of long-term therapy and increases the cost of health services. We propose in this paper different models of patient drug consumption in breast cancer treatments. The aim of these different approaches is to predict medication non-adherence while giving insights to doctors of the underlying reasons of these illegitimate drop-outs. Working with oncologists, we show the interest of Machine- Learning algorithms fined tune by the feedback of experts to estimate a risk score of a patient's non-adherence and thus improve support throughout their care path.
ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness
Geirhos, Robert, Rubisch, Patricia, Michaelis, Claudio, Bethge, Matthias, Wichmann, Felix A., Brendel, Wieland
Convolutional Neural Networks (CNNs) are commonly thought to recognise objects by learning increasingly complex representations of object shapes. Some recent studies hint to a more important role of image textures. We here put these conflicting hypotheses to a quantitative test by evaluating CNNs and human observers on images with a texture-shape cue conflict. We show that ImageNet-trained CNNs are strongly biased towards recognising textures rather than shapes, which is in stark contrast to human behavioural evidence and reveals fundamentally different classification strategies. We then demonstrate that the same standard architecture (ResNet-50) that learns a texture-based representation on ImageNet is able to learn a shape-based representation instead when trained on "Stylized-ImageNet", a stylized version of ImageNet. This provides a much better fit for human behavioural performance in our well-controlled psychophysical lab setting (nine experiments totalling 48,560 psychophysical trials across 97 observers) and comes with a number of unexpected emergent benefits such as improved object detection performance and previously unseen robustness towards a wide range of image distortions, highlighting advantages of a shape-based representation.
The Looming Rise of AI-Powered Malware
In the past two years, we've learned that machine learning algorithms can manipulate public opinion, cause fatal car crashes, create fake porn, and manifest extremely sexist and racist behavior. And now, the cybersecurity threats of deep learning and neural networks are emerging. We're just beginning to catch glimpses of a future in which cybercriminals trick neural networks into making fatal mistakes and use deep learning to hide their malware and find their target among millions of users. Part of the challenge of securing artificial intelligence applications lies in the fact it's hard to explain how they work, and even the people who create them are often hard-pressed to make sense of their inner workings. But unless we prepare ourselves for what is to come, we'll learn to appreciate and react to these threats the hard way.
Understanding the power of deep learning: A look this technology
Deep learning is a concept that most people know is valuable but do not understand. Global industries, however, are seeing it's worth and investing heavily in the technology; the global deep learning market is expected to reach $10.2 billion by 2025. What are the nuances and intricacies that make deep learning a practical solution to some of today's most complex problems? And how can this technology be understood by more, making it a less intimidating and more approachable topic? "One reason why deep learning is a confusing concept to many is that it is often used alongside the terms machine learning (ML) and artificial intelligence (AI). Deep learning (DL) is a subset of ML, which is itself a subset of AI," explains Jennifer Roubaud, the VP of UK and Ireland for Dataiku.
Why AI and deep learning are the perfect tools to help us understand the past
The world of academia is generally not known for being on the cutting edge of technology. However, as technology rapidly advances, historians and researchers can utilize both artificial intelligence (AI) and deep learning to make their jobs easier. Artificial intelligence is the ability for a machine to imitate intelligent human behavior. Deep learning, a subset of machine learning, is the intermediary between machine learning and neural networks. Deep learning provides a fast and relatively easy way to process massive amounts of data, much of which would be tedious and time consuming for a human to process; because of this, pattern recognition is one of deep learning's greatest strengths. Historians are finding that the practical applications of deep learning can help them work with large amounts of information much easier.
Cluster-Based Learning from Weakly Labeled Bags in Digital Pathology
Akbar, Shazia, Martel, Anne L.
To alleviate the burden of gathering detailed expert annotations when training deep neural networks, we propose a weakly supervised learning approach to recognize metastases in microscopic images of breast lymph nodes. We describe an alternative training loss which clusters weakly labeled bags in latent space to inform relevance of patch-instances during training of a convolutional neural network. We evaluate our method on the Camelyon dataset which contains high-resolution digital slides of breast lymph nodes, where labels are provided at the image-level and only subsets of patches are made available during training.