On the Web, there is always a need to aggregate opinions from the crowd (as in posts, social networks, forums, etc.). Different mechanisms have been implemented to capture these opinions such as "Like" in Facebook, "Favorite" in Twitter, thumbs-up/down, flagging, and so on. However, in more contested domains (e.g. Wikipedia, political discussion, and climate change discussion) these mechanisms are not sufficient since they only deal with each issue independently without considering the relationships between different claims. We can view a set of conflicting arguments as a graph in which the nodes represent arguments and the arcs between these nodes represent the defeat relation. A group of people can then collectively evaluate such graphs. To do this, the group must use a rule to aggregate their individual opinions about the entire argument graph. Here, we present the first experimental evaluation of different principles commonly employed by aggregation rules presented in the literature. We use randomized controlled experiments to investigate which principles people consider better at aggregating opinions under different conditions. Our analysis reveals a number of factors, not captured by traditional formal models, that play an important role in determining the efficacy of aggregation. These results help bring formal models of argumentation closer to real-world application.

We present an approach to reasoning with strict and defeasible rules over literals. A controlled natural language is employed as human/machine interface to facilitate the specification of knowledge and verbalization of results. Reasoning on the rules is done by a direct semantics that addresses several issues for current approaches to argumentation-based defeasible reasoning. Techniques from formal argumentation theory are employed to justify conclusions of the approach; therefore, we not only address automated reasoning but also human acceptance of provided conclusions.

As autonomous intelligent systems become more widespread, society is beginning to ask: "What are the machines up to?". Various forms of artificial intelligence control our latest cars, load balance components of our power grids, dictate much of the movement in our stock markets and help doctors diagnose and treat our ailments. As they become increasingly able to learn and model more complex phenomena, so the ability of human users to understand the reasoning behind their decisions often decreases. It becomes very difficult to ensure that the robot will perform properly and that it is possible to correct errors. In this paper, we outline a variety of techniques for generating the underlying knowledge required for explainable artificial intelligence, ranging from early work in expert systems through to systems based on Behavioural Cloning. These are techniques that may be used to build intelligent robots that explain their decisions and justify their actions. We will then illustrate how decision trees are particularly well suited to generating these kinds of explanations. We will also discuss how additional explanations can be obtained, beyond simply the structure of the tree, based on knowledge of how the training data was generated. Finally, we will illustrate these capabilities in the context of a robot learning to drive over rough terrain in both simulation and in reality.

In [3] we presented a prototype of a system that enables users to explore arguments for a given topic. This involves these steps: 1. Argument identification. In the first step, arguments concerning a given topic are identified in a given text and attacking and supporting relationships between the propositions are established. The result is an argumentation graph. In the future we hope to use argumentation mining techniques to automate this step. At this time, this is done manually by marking up some text.

The effort aims to better understand how a machine-learning model comes to a logical conclusion.

In existing literature, while approximate approaches based on Monte-Carlo simulation technique have been proposed to compute the semantics of probabilistic argumentation, how to improve the efficiency of computation without using simulation technique is still an open problem. In this paper, we address this problem from the following two perspectives. First, conceptually, we define specific properties to characterize the subgraphs of a PrAG with respect to a given extension, such that the probability of a set of arguments E being an extension can be defined in terms of these properties, without (or with less) construction of subgraphs. Second, computationally, we take preferred semantics as an example, and develop algorithms to evaluate the efficiency of our approach. The results show that our approach not only dramatically decreases the time for computing p(E^\sigma), but also has an attractive property, which is contrary to that of existing approaches: the denser the edges of a PrAG are or the bigger the size of a given extension E is, the more efficient our approach computes p(E^\sigma). Meanwhile, it is shown that under complete and preferred semantics, the problems of determining p(E^\sigma) are fixed-parameter tractable.

Crowd workers are human and thus sometimes make mistakes. In order to ensure the highest quality output, requesters often issue redundant jobs with gold test questions and sophisticated aggregation mechanisms based on expectation maximization (EM). While these methods yield accurate results in many cases, they fail on extremely difficult problems with local minima, such as situations where the majority of workers get the answer wrong. Indeed, this has caused some researchers to conclude that on some tasks crowdsourcing can never achieve high accuracies, no matter how many workers are involved. This paper presents a new quality-control workflow, called MicroTalk, that requires some workers to Justify their reasoning and asks others to Reconsider their decisions after reading counter-arguments from workers with opposing views. Experiments on a challenging NLP annotation task with workers from Amazon Mechanical Turk show that (1) argumentation improves the accuracy of individual workers by 20%, (2) restricting consideration to workers with complex explanations improves accuracy even more, and (3) our complete MicroTalk aggregation workflow produces much higher accuracy than simpler voting approaches for a range of budgets.

To gain intuition and reasoning of a model is to have understanding and trust--transparency. When you strike a nail with a hammer, it's pretty predictable what might happen: the nail could get hit, the hammer could miss, or very rarely, the hammer's head may fly off of the handle. When you replace the hammer with a black box that works correctly 99.999% of the time, but for 0.001%, something completely unpredictable happens, then there's a problem with volatility because that unpredictable event may have unacceptable consequences. I think explainable AI could help with intuitive and more fine-grained risk analysis, and that's certainly a good thing in high-stakes applications such as defense.

The goal of Explainable AI (XAI) is to create a suite of new or modified machine learning techniques that produce explainable models that, when combined with effective explanation techniques, enable end users to understand, appropriately trust, and effectively manage the emerging generation of AI systems.

Dramatic success in machine learning has led to a torrent of Artificial Intelligence (AI) applications. Continued advances promise to produce autonomous systems that will perceive, learn, decide, and act on their own. However, the effectiveness of these systems is limited by the machine's current inability to explain their decisions and actions to human users. The Department of Defense is facing challenges that demand more intelligent, autonomous, and symbiotic systems. Explainable AI--especially explainable machine learning--will be essential if future warfighters are to understand, appropriately trust, and effectively manage an emerging generation of artificially intelligent machine partners.