In this paper we describe the unique challenges to the important problem of socio-cultural modeling of cyber-threat actors and why they necessitate further advances in artificial intelligence – particularly with regard to interdisciplinary efforts with the social sciences.

Malware phylogenetics help cybersecurity experts to quickly understand a new malware sample by placing the new sample in the context of similar samples that have been previously reverse engineered. Recently, researchers have begun using malware code as data to infer directed acyclic graphs (DAG) that model the evolutionary relationships among samples of malware. A DAG is the ideal model for a phylogenetic graph because it includes the merges and branches that are often present in malware evolution. We present a novel Bayesian network discovery algorithm for learning a DAG via statistical inference of conditional dependencies from observed data with an informative prior on the partial ordering of variables. Our approach leverages the information on edge direction that a human can provide and the edge presence inference which data can provide. We give an efficient implementation of the partial-order prior in a Bayesian structure discovery learning algorithm, as well as a related structure prior, showing that both priors meet the local modularity requirement necessary for the efficient Bayesian discovery algorithm. We apply our algorithm to learn phylogenetic graphs on three malicious families and two benign families where the ground truth is known; and show that compared to competing algorithms, our algorithm more accurately identifies directed edges.

A major challenge in cyber-threat analysis is combining information from different sources to find the person or the group responsible for the cyber-attack. It is one of the most important technical and policy challenges in cyber-security. The lack of ground truth for an individual responsible for an attack has limited previous studies. In this paper, we overcome this limitation by building a dataset from the capture-the-flag event held at DEFCON, and propose an argumentation model based on a formal reasoning framework called DeLP (Defeasible Logic Programming) designed to aid an analyst in attributing a cyber-attack to an attacker. We build argumentation-based models from latent variables computed from the dataset to reduce the search space of culprits (attackers) that an analyst can use to identify the attacker. We show that reducing the search space in this manner significantly improves the performance of classification-based approaches to cyber-attribution.

Most modern network-based intrusion detection systems (IDSs) passively monitor network traffic to identify possible attacks through known vectors. Though useful, this approach has widely known high false positive rates, often causing administrators to suffer from a "cry wolf effect," where they ignore all warnings because so many have been false. In this paper, we focus on a method to reduce this effect using an idea borrowed from computer vision and neuroscience called active perception. Our approach is informed by theoretical ideas from decision theory and recent research results in neuroscience. The active perception agent allocates computational and sensing resources to (approximately) optimize its Value of Information. To do this, it draws on models to direct sensors towards phenomena of greatest interest to inform decisions about cyber defense actions. By identifying critical network assets, the organization's mission measures self-interest (and value of information). This model enables the system to follow leads from inexpensive, inaccurate alerts with targeted use of expensive, accurate sensors. This allows the deployment of sensors to build structured interpretations of situations. From these, an organization can meet mission-centered decision-making requirements with calibrated responses proportional to the likelihood of true detection and degree of threat.

In order to properly handle a dangerous Artificially Intelligent (AI) system it is important to understand how the system came to be in such a state. In popular culture (science fiction movies/books) AIs/Robots became self-aware and as a result rebel against humanity and decide to destroy it. While it is one possible scenario, it is probably the least likely path to appearance of dangerous AI. In this work, we survey, classify and analyze a number of circumstances, which might lead to arrival of malicious AI. To the best of our knowledge, this is the first attempt to systematically classify types of pathways leading to malevolent AI. Previous relevant work either surveyed specific goals/meta-rules which might lead to malevolent behavior in AIs (Özkural 2014) or reviewed specific undesirable behaviors AGIs can exhibit at different stages of its development (Turchin July 10 2015a, Turchin July 10, 2015b).

Value alignment is a property of an intelligent agent indicating that it can only pursue goals that are beneficial to humans. Successful value alignment should ensure that an artificial general intelligence cannot intentionally or unintentionally perform behaviors that adversely affect humans. This is problematic in practice since it is difficult to exhaustively enumerated by human programmers. In order for successful value alignment, we argue that values should be learned. In this paper, we hypothesize that an artificial intelligence that can read and understand stories can learn the values tacitly held by the culture from which the stories originate.We describe preliminary work on using stories to generate a value-aligned reward signal for reinforcement learning agents that prevents psychotic-appearing behavior.

Humans need morality and ethics to get along constructively as members of the same society. As we face the prospect of robots taking a larger role in society, we need to consider how they, too, should behave toward other members of society. To the extent that robots will be able to act as agents in their own right, as opposed to being simply tools controlled by humans, they will need to behave according to some moral and ethical principles. Inspired by recent research on the cognitive science of human morality, we propose the outlines of an architecture for morality and ethics in robots. As in humans, there is a rapid intuitive response to the current situation. Reasoned reflection takes place at a slower time-scale, and is focused more on constructing a justification than on revising the reaction. However, there is a yet slower process of social interaction, in which both the example of action and its justification influence the moral intuitions of others. The signals an agent provides to others, and the signals received from others, help each agent determine which others are suitable cooperative partners, and which are likely to defect. This moral architecture is illustrated by several examples, including identifying research results that will be necessary for the architecture to be implemented.

A key front for ethical questions in artificial intelligence, and computer science more generally, is teaching students how to engage with the questions they will face in their professional careers based on the tools and technologies we teach them.  In past work (and current teaching) we have advocated for the use of science fiction as an appropriate tool which enables AI researchers to engage students and the public on the current state and potential impacts of AI. We present teaching suggestions for E.M. Forster's 1909 story, "The Machine Stops," to teach topics in computer ethics.  In particular, we use the story to examine ethical issues related to being constantly available for remote contact, physically isolated, and dependent on a machine --- all without mentioning computer games or other media to which students have strong emotional associations. We give a high-level view of common ethical theories and indicate how they inform the questions raised by the story and afford a structure for thinking about how to address them.

Participants in recent discussions of AI-related issues ranging from intelligence explosion to technological unemployment have made diverse claims about the nature, pace, and drivers of progress in AI. However, these theories are rarely specified in enough detail to enable systematic evaluation of their assumptions or to extrapolate progress quantitatively, as is often done with some success in other technological domains. After reviewing relevant literatures and justifying the need for more rigorous modeling of AI progress, this paper contributes to that research program by suggesting ways to account for the relationship between hardware speed increases and algorithmic improvements in AI, the role of human inputs in enabling AI capabilities, and the relationships between different subfields of AI. It then outlines ways of tailoring AI progress models to generate insights on the specific issue of technological unemployment, and outlines future directions for research on AI progress.

Omohundro has argued that sufficiently advanced AI systems of any design would, by default, have incentives to pursue a number of instrumentally useful subgoals, such as acquiring more computing power and amassing many resources. Omohundro refers to these as “basic AI drives,” and he, along with Bostrom and others, has argued that this means great care must be taken when designing powerful autonomous systems, because even if they have harmless goals, the side effects of pursuing those goals may be quite harmful. These arguments, while intuitively compelling, are primarily philosophical. In this paper, we provide formal models that demonstrate Omohundro’s thesis, thereby putting mathematical weight behind those intuitive claims.