Interpretability in machine learning (ML) is crucial for high stakes decisions and troubleshooting. In this work, we provide fundamental principles for interpretable ML, and dispel common misunderstandings that dilute the importance of this crucial topic. We also identify 10 technical challenge areas in interpretable machine learning and provide history and background on each problem. Some of these problems are classically important, and some are recent problems that have arisen in the last few years. These problems are: (1) Optimizing sparse logical models such as decision trees; (2) Optimization of scoring systems; (3) Placing constraints into generalized additive models to encourage sparsity and better interpretability; (4) Modern case-based reasoning, including neural networks and matching for causal inference; (5) Complete supervised disentanglement of neural networks; (6) Complete or even partial unsupervised disentanglement of neural networks; (7) Dimensionality reduction for data visualization; (8) Machine learning models that can incorporate physics and other generative or causal constraints; (9) Characterization of the "Rashomon set" of good models; and (10) Interpretable reinforcement learning. This survey is suitable as a starting point for statisticians and computer scientists interested in working in interpretable machine learning.

Deep Neural Networks (DNNs) have shown great success in completing complex tasks. However, DNNs inevitably bring high computational cost and storage consumption due to the complexity of hierarchical structures, thereby hindering their wide deployment in Internet-of-Things (IoT) devices, which have limited computational capability and storage capacity. Therefore, it is a necessity to investigate the technologies to compact DNNs. Despite tremendous advances in compacting DNNs, few surveys summarize compacting-DNNs technologies, especially for IoT applications. Hence, this paper presents a comprehensive study on compacting-DNNs technologies. We categorize compacting-DNNs technologies into three major types: 1) network model compression, 2) Knowledge Distillation (KD), 3) modification of network structures. We also elaborate on the diversity of these approaches and make side-by-side comparisons. Moreover, we discuss the applications of compacted DNNs in various IoT applications and outline future directions.

As the use of deep learning techniques has grown across various fields over the past decade, complaints about the opaqueness of the black-box models have increased, resulting in an increased focus on transparency in deep learning models. This work investigates various methods to improve the interpretability of deep neural networks for natural language processing (NLP) tasks, including machine translation and sentiment analysis. We provide a comprehensive discussion on the definition of the term \textit{interpretability} and its various aspects at the beginning of this work. The methods collected and summarised in this survey are only associated with local interpretation and are divided into three categories: 1) explaining the model's predictions through related input features; 2) explaining through natural language explanation; 3) probing the hidden states of models and word representations.

Subspace clustering is an important unsupervised clustering approach. It is based on the assumption that the high-dimensional data points are approximately distributed around several low-dimensional linear subspaces. The majority of the prominent subspace clustering algorithms rely on the representation of the data points as linear combinations of other data points, which is known as a self-expressive representation. To overcome the restrictive linearity assumption, numerous nonlinear approaches were proposed to extend successful subspace clustering approaches to data on a union of nonlinear manifolds. In this comparative study, we provide a comprehensive overview of nonlinear subspace clustering approaches proposed in the last decade. We introduce a new taxonomy to classify the state-of-the-art approaches into three categories, namely locality preserving, kernel based, and neural network based. The major representative algorithms within each category are extensively compared on carefully designed synthetic and real-world data sets. The detailed analysis of these approaches unfolds potential research directions and unsolved challenges in this field.

Recent years have witnessed rapid growth in satellite-based approaches to quantifying aspects of land use, especially those monitoring the outcomes of sustainable development programs. Burke et al. reviewed this recent progress with a particular focus on machine-learning approaches and artificial intelligence methods. Drawing on examples mostly from Africa, they conclude that satellite-based methods enhance rather than replace ground-based data collection, and progress depends on a combined approach. Science , this issue p. [eabe8628][1] ### BACKGROUND Accurate and comprehensive measurements of a range of sustainable development outcomes are fundamental inputs into both research and policy. For instance, good measures are needed to monitor progress toward sustainability goals and evaluate interventions designed to improve development outcomes. Traditional approaches to measurement of many key outcomes rely on household surveys that are conducted infrequently in many parts of the world and are often of low accuracy. The paucity of ground data stands in contrast to the rapidly growing abundance and quality of satellite imagery. Multiple public and private sensors launched in recent years provide temporal, spatial, and spectral information on changes happening on Earth’s surface. Here we review a rapidly growing scientific literature that seeks to use this satellite imagery to measure and understand various outcomes related to sustainable development. We pay particular attention to recent approaches that use methods from artificial intelligence to extract information from images, as these methods typically outperform earlier approaches and enable new insights. Our focus is on settings and applications where humans themselves, or what they produce, are the outcome of interest and on where these outcomes are being measured using satellite imagery. ### ADVANCES We describe and synthesize the variety of approaches that have been used to extract information from satellite imagery, with particular attention given to recent machine learning–based approaches and settings in which training data are limited or noisy. We then quantitatively assess predictive performance of these approaches in the domains of smallholder agriculture, economic livelihoods, population, and informal settlements. We show that satellite-based performance in predicting these outcomes is reasonably strong and improving. Performance improvements have come through a combination of more numerous and accurate training data, more abundant and higher-quality imagery, and creative application of advances in computer vision to satellite inputs and sustainability outcomes. Further, our analyses suggest that reported model performance likely understates true performance in many settings, given the noisy data on which predictions are evaluated and the types of noise typically observed in sustainability applications. For multiple outcomes of interest, satellite-based estimates can now equal or exceed the accuracy of traditional approaches to outcome measurement. We describe multiple methods through which the true performance of satellite-based approaches can be better understood. Integration of satellite-based sustainability measurements into research has been broad, and we describe applications in agriculture, fisheries, health, and economics. Documented uses of these measurements in public-sector decision-making are rarer, which we attribute in part to the novelty of the approaches, their lack of interpretability, and the potential benefits to some policy-makers of not having certain outcomes be measured. ### OUTLOOK The largest constraint to satellite-based model performance is now training data rather than imagery. While imagery has become abundant, the scarcity and frequent unreliability of ground data make both training and validation of satellite-based models difficult. Expanding the quantity and quality of such data will quickly accelerate progress in this field. Other opportunities for advancement include improvements in model interpretability, fusion of satellites with other nontraditional data that provide complementary information, and more-rigorous evaluation of satellite-based approaches (relative to available alternatives) in the context of specific use cases. Nevertheless, despite the current and future promise of satellite-based approaches, we argue that these approaches will amplify rather than replace existing ground-based data collection efforts in most settings. Many outcomes of interest will likely never be accurately estimated with satellites; for outcomes where satellites do have predictive power, high-quality local training data can nearly always improve model performance. ![Figure][2] Increasing collection of satellite imagery can help measure livelihood outcomes in areas where ground data are sparse. (Left) Interval between nationally representative economic surveys over the past three decades shows long lags in many developing countries. (Middle) Recently added public and private satellites have broken the traditional trade-off between temporal and spatial resolution. (Right) Performance in measuring the presence of informal settlements, crop yields on smallholder agricultural plots, and village-level asset wealth. R 2, coefficient of determination. Accurate and comprehensive measurements of a range of sustainable development outcomes are fundamental inputs into both research and policy. We synthesize the growing literature that uses satellite imagery to understand these outcomes, with a focus on approaches that combine imagery with machine learning. We quantify the paucity of ground data on key human-related outcomes and the growing abundance and improving resolution (spatial, temporal, and spectral) of satellite imagery. We then review recent machine learning approaches to model-building in the context of scarce and noisy training data, highlighting how this noise often leads to incorrect assessment of model performance. We quantify recent model performance across multiple sustainable development domains, discuss research and policy applications, explore constraints to future progress, and highlight research directions for the field. [1]: /lookup/doi/10.1126/science.abe8628 [2]: pending:yes

Digi-Tech Pharma & AI Virtual Conference & Expo Date: 05-06 May 2021 (10:00-18:30 BST) Location: An online and virtual event The 4th Annual Digi-Tech Pharma & AI conference brings with it even more interactive sessions, expert speakers, senior professionals and decision makers from leading pharma, bio-tech and healthcare industry. Meet the decision makers, benchmark and learn from real-life use cases to drive organizational change and to understand the new cutting-edge technologies and practical solutions. In this 4th edition as we explore the novel technologies and developments reforming pharmaceutical industry, we also dive deep into the implementation and advances in machine learning, deep learning, artificial intelligence, informatics and data science which has redefined the development of new drugs, tackle diseases, improving healthcare and much more. The enhancements in data management and data integration are providing improvements to both the speed and quality of drug discovery and many clinical trial processes. To be in the forefront, a necessity for partnership and collaboration with healthcare provider is a must for the pharmaceutical companies, and these partnerships will also lead to massive advances in R&D using artificial intelligence in genomics and precision medicine to develop a deep understanding of the root causes of diseases.

This work provides a starting point for researchers interested in gaining a deeper understanding of the big picture of artificial intelligence (AI). To this end, a narrative is conveyed that allows the reader to develop an objective view on current developments that is free from false promises that dominate public communication. An essential takeaway for the reader is that AI must be understood as an umbrella term encompassing a plethora of different methods, schools of thought, and their respective historical movements. Consequently, a bottom-up strategy is pursued in which the field of AI is introduced by presenting various aspects that are characteristic of the subject. This paper is structured in three parts: (i) Discussion of current trends revealing false public narratives, (ii) an introduction to the history of AI focusing on recurring patterns and main characteristics, and (iii) a critical discussion on the limitations of current methods in the context of the potential emergence of a strong(er) AI. It should be noted that this work does not cover any of these aspects holistically; rather, the content addressed is a selection made by the author and subject to a didactic strategy.

Federated learning has emerged as a popular technique for distributing machine learning (ML) model training across the wireless edge. In this paper, we propose two timescale hybrid federated learning (TT-HF), which is a hybrid between the device-to-server communication paradigm in federated learning and device-to-device (D2D) communications for model training. In TT-HF, during each global aggregation interval, devices (i) perform multiple stochastic gradient descent iterations on their individual datasets, and (ii) aperiodically engage in consensus formation of their model parameters through cooperative, distributed D2D communications within local clusters. With a new general definition of gradient diversity, we formally study the convergence behavior of TT-HF, resulting in new convergence bounds for distributed ML. We leverage our convergence bounds to develop an adaptive control algorithm that tunes the step size, D2D communication rounds, and global aggregation period of TT-HF over time to target a sublinear convergence rate of O(1/t) while minimizing network resource utilization. Our subsequent experiments demonstrate that TT-HF significantly outperforms the current art in federated learning in terms of model accuracy and/or network energy consumption in different scenarios where local device datasets exhibit statistical heterogeneity.

In recent years, researchers have proposed many deep learning (DL) methods for various tasks, and particularly face recognition (FR) made an enormous leap using these techniques. Deep FR systems benefit from the hierarchical architecture of the DL methods to learn discriminative face representation. Therefore, DL techniques significantly improve state-of-the-art performance on FR systems and encourage diverse and efficient real-world applications. In this paper, we present a comprehensive analysis of various FR systems that leverage the different types of DL techniques, and for the study, we summarize 168 recent contributions from this area. We discuss the papers related to different algorithms, architectures, loss functions, activation functions, datasets, challenges, improvement ideas, current and future trends of DL-based FR systems. We provide a detailed discussion of various DL methods to understand the current state-of-the-art, and then we discuss various activation and loss functions for the methods. Additionally, we summarize different datasets used widely for FR tasks and discuss challenges related to illumination, expression, pose variations, and occlusion. Finally, we discuss improvement ideas, current and future trends of FR tasks.

Real-world decision-making tasks are generally complex, requiring trade-offs between multiple, often conflicting, objectives. Despite this, the majority of research in reinforcement learning and decision-theoretic planning either assumes only a single objective, or that multiple objectives can be adequately handled via a simple linear combination. Such approaches may oversimplify the underlying problem and hence produce suboptimal results. This paper serves as a guide to the application of multi-objective methods to difficult problems, and is aimed at researchers who are already familiar with single-objective reinforcement learning and planning methods who wish to adopt a multi-objective perspective on their research, as well as practitioners who encounter multi-objective decision problems in practice. It identifies the factors that may influence the nature of the desired solution, and illustrates by example how these influence the design of multi-objective decision-making systems for complex problems.