Contrastive explanations clarify why an event occurred in contrast to another. They are more inherently intuitive to humans to both produce and comprehend. We propose a methodology to produce contrastive explanations for classification models by modifying the representation to disregard non-contrastive information, and modifying model behavior to only be based on contrastive reasoning. Our method is based on projecting model representation to a latent space that captures only the features that are useful (to the model) to differentiate two potential decisions. We demonstrate the value of contrastive explanations by analyzing two different scenarios, using both high-level abstract concept attribution and low-level input token/span attribution, on two widely used text classification tasks. Specifically, we produce explanations for answering: for which label, and against which alternative label, is some aspect of the input useful? And which aspects of the input are useful for and against particular decisions? Overall, our findings shed light on the ability of label-contrastive explanations to provide a more accurate and finer-grained interpretability of a model's decision.

Continual Learning (CL) algorithms have recently received a lot of attention as they attempt to overcome the need to train with an i.i.d. sample from some unknown target data distribution. Building on prior work, we study principled ways to tackle the CL problem by adopting a Bayesian perspective and focus on continually learning a task-specific posterior distribution via a shared meta-model, a task-conditioned hypernetwork. This approach, which we term Posterior-replay CL, is in sharp contrast to most Bayesian CL approaches that focus on the recursive update of a single posterior distribution. The benefits of our approach are (1) an increased flexibility to model solutions in weight space and therewith less susceptibility to task dissimilarity, (2) access to principled task-specific predictive uncertainty estimates, that can be used to infer task identity during test time and to detect task boundaries during training, and (3) the ability to revisit and update task-specific posteriors in a principled manner without requiring access to past data. The proposed framework is versatile, which we demonstrate using simple posterior approximations (such as Gaussians) as well as powerful, implicit distributions modelled via a neural network. We illustrate the conceptual advance of our framework on low-dimensional problems and show performance gains on computer vision benchmarks.

One of the increasingly important technologies dealing with the growing complexity of the digitalization of almost all human activities is Artificial intelligence, more precisely machine learning Despite the fact, that we live in a Big data world where almost everything is digitally stored, there are many real-world situations, where researchers are faced with small data samples. The present study aim is to answer the following research question namely What is the small data problem in machine learning and how it is solved?. Our bibliometric study showed a positive trend in the number of research publications concerning the use of small datasets and substantial growth of the research community dealing with the small dataset problem, indicating that the research field is moving toward higher maturity levels. Despite notable international cooperation, the regional concentration of research literature production in economically more developed countries was observed.

Both Data Science and machine learning are very inter-related. It's hard to distinguish them at least at the Masters level. So, don't bother to differentiate them. Between data science and data analytics, it all depends on your existing skills and learning objectives. The course pattern, structure, syllabus everything is quite similar and is interrelated.

Artificial intelligence has maintained a steady graph of growth in the past few years. But the pandemic led to a rapid digital transformation, which further prompted rapid innovation in the realm. As per McKinsey's State of AI survey published in November 2020, half of the survey respondents had stated their companies had adopted AI in at least one function. Experts well versed in the AI domain predict that it'll continue to experience vast expansion and development in meaningful ways in 2021 and beyond. Let's mull over some of the developments in the domain of AI that you can expect.

Active learning is an area of modern machine learning that studies how data points can be sequentially selected for labeling to train a model with as few labeled examples as possible (Settles, 2009). Minimizing the number of labeled examples is critical in any learning scenario where labels are expensive to obtain, such as in healthcare applications where a medical expert must hand-label each training example (Liu, 2004), or where only a limited number of examples can be evaluated, such as in drug discovery (Warmuth et al., 2003). The active selection of data points shares many technical parallels with channel coding with feedback, where a message is encoded into a sequence of symbols transmitted across a noisy channel and each symbol is selected based on the message and past channel outputs. In active learning, the optimal classifier parameters play the role of the "message" while the sequence of examples with noisy labels plays the role of "channel outputs" available through feedback to select the next example for labeling. Both feedback channel coding and active learning seek to minimize the number of encoder actions, leverage a history of noisy observations to select the next most informative action, must account for observation noise, and should operate in a computationally efficient manner. Although there exists a large literature studying the intersection of information theory with machine learning (Xu and Raginsky, 2017) and specifically active learning (Naghshvar et al., 2015), there remain open questions about the best ways to directly leverage techniques in channel coding for active example selection. The main contribution of this work is a formulation of general active learning problems in terms of a feedback coding system, and a demonstration of this approach through the application and analysis of active learning in logistic regression. To motivate this approach, we first examine active learning through the lens of feedback channel coding by identifying communications system components, including a deterministic encoder, noisy channel, channel input constraints, and capacity-achieving distribution. With these components identified, we show how typical structural constraints in active learning problems prevent the direct application of existing feedback coding approaches such as posterior matching (Ma and Coleman, 2011).

The societal and ethical implications of the use of opaque artificial intelligence systems for consequential decisions, such as welfare allocation and criminal justice, have generated a lively debate among multiple stakeholder groups, including computer scientists, ethicists, social scientists, policy makers, and end users. However, the lack of a common language or a multi-dimensional framework to appropriately bridge the technical, epistemic, and normative aspects of this debate prevents the discussion from being as productive as it could be. Drawing on the philosophical literature on the nature and value of explanations, this paper offers a multi-faceted framework that brings more conceptual precision to the present debate by (1) identifying the types of explanations that are most pertinent to artificial intelligence predictions, (2) recognizing the relevance and importance of social and ethical values for the evaluation of these explanations, and (3) demonstrating the importance of these explanations for incorporating a diversified approach to improving the design of truthful algorithmic ecosystems. The proposed philosophical framework thus lays the groundwork for establishing a pertinent connection between the technical and ethical aspects of artificial intelligence systems.

Abstract--In neural networks, developing regularization algorithm s to settle overfitting is one of the major study areas. We prop ose a new approach for the regularization of neural networks by th e local Rademacher complexity called LocalDrop. A new regul arization function for both fully-connected networks (FCNs) and conv olutional neural networks (CNNs), including drop rates and weight matrices, has been developed based on the proposed upper bound of the lo cal Rademacher complexity by the strict mathematical deduc tion. The analyses of dropout in FCNs and DropBlock in CNNs with kee p rate matrices in different layers are also included in the c omplexity analyses. With the new regularization function, we establi sh a two-stage procedure to obtain the optimal keep rate matr ix and weight matrix to realize the whole training model. Extensive exper iments have been conducted to demonstrate the effectivenes s of LocalDrop in different models by comparing it with several algorithms and the effects of different hyperparameters on the final per formances. Neural networks have lately shown impressive performance i n sophisticated real-world situations, including image cla ssification [1], object recognition [2] and image captioning [3]. Low, m iddle and high level features are integrated into deep neural netw orks, which are usually trained in an end-to-end manner.

Explainable NLP (ExNLP) has increasingly focused on collecting human-annotated explanations. These explanations are used downstream in three ways: as data augmentation to improve performance on a predictive task, as a loss signal to train models to produce explanations for their predictions, and as a means to evaluate the quality of model-generated explanations. In this review, we identify three predominant classes of explanations (highlights, free-text, and structured), organize the literature on annotating each type, point to what has been learned to date, and give recommendations for collecting ExNLP datasets in the future.

The typical ingredient-tetris bottleneck played between guest and server while dining out has amplified during COVID-19. Growth in online ordering and takeout has prompted customers with dietary needs to search online for dietary answers more than ever before.1 With over 52% of Americans following at least one diet, and less than 10% of restaurants labeling dietary information (typically not exhaustive), the information gap has never been wider. Prompted by an Ulcerative Colitis health scare for co-founder Tamir Barzilai, Honeycomb.ai is set on eliminating the frustrating process of manual menu parsing by creating a portal for anyone with dietary needs to find suitable food to eat. "After my personal diagnosis, I realized how many others struggle with finding food to eat due to a variety of reasons. The lack of ubiquitous dietary and ingredient transparency didn't make sense from both consumer and business perspectives," says Barzilai.