A linear non-Gaussian structural equation model called LiNGAM is an identifiable model for exploratory causal analysis. Previous methods estimate a causal ordering of variables and their connection strengths based on a single dataset. However, in many application domains, data are obtained under different conditions, that is, multiple datasets are obtained rather than a single dataset. In this paper, we present a new method to jointly estimate multiple LiNGAMs under the assumption that the models share a causal ordering but may have different connection strengths and differently distributed variables. In simulations, the new method estimates the models more accurately than estimating them separately.

In the context of clustering, we assume a generative model where each cluster is the result of sampling points in the neighborhood of an embedded smooth surface; the sample may be contaminated with outliers, which are modeled as points sampled in space away from the clusters. We consider a prototype for a higher-order spectral clustering method based on the residual from a local linear approximation. We obtain theoretical guarantees for this algorithm and show that, in terms of both separation and robustness to outliers, it outperforms the standard spectral clustering algorithm (based on pairwise distances) of Ng, Jordan and Weiss (NIPS '01). The optimal choice for some of the tuning parameters depends on the dimension and thickness of the clusters. We provide estimators that come close enough for our theoretical purposes. We also discuss the cases of clusters of mixed dimensions and of clusters that are generated from smoother surfaces. In our experiments, this algorithm is shown to outperform pairwise spectral clustering on both simulated and real data.

Many problems in artificial intelligence require adaptively making a sequence of decisions with uncertain outcomes under partial observability. Solving such stochastic optimization problems is a fundamental but notoriously difficult challenge. In this paper, we introduce the concept of adaptive submodularity, generalizing submodular set functions to adaptive policies. We prove that if a problem satisfies this property, a simple adaptive greedy algorithm is guaranteed to be competitive with the optimal policy. In addition to providing performance guarantees for both stochastic maximization and coverage, adaptive submodularity can be exploited to drastically speed up the greedy algorithm by using lazy evaluations. We illustrate the usefulness of the concept by giving several examples of adaptive submodular objectives arising in diverse AI applications including management of sensing resources, viral marketing and active learning. Proving adaptive submodularity for these problems allows us to recover existing results in these applications as special cases, improve approximation guarantees and handle natural generalizations.

This paper illustrates successful implementation of three evolutionary algorithms, namely- Particle Swarm Optimization(PSO), Artificial Bee Colony (ABC) and Bacterial Foraging Optimization (BFO) algorithms to economic load dispatch problem (ELD). Power output of each generating unit and optimum fuel cost obtained using all three algorithms have been compared. The results obtained show that ABC and BFO algorithms converge to optimal fuel cost with reduced computational time when compared to PSO for the two example problems considered.

Machine learning approaches to multi-label document classification have to date largely relied on discriminative modeling techniques such as support vector machines. A drawback of these approaches is that performance rapidly drops off as the total number of labels and the number of labels per document increase. This problem is amplified when the label frequencies exhibit the type of highly skewed distributions that are often observed in real-world datasets. In this paper we investigate a class of generative statistical topic models for multi-label documents that associate individual word tokens with different labels. We investigate the advantages of this approach relative to discriminative models, particularly with respect to classification problems involving large numbers of relatively rare labels. We compare the performance of generative and discriminative approaches on document labeling tasks ranging from datasets with several thousand labels to datasets with tens of labels. The experimental results indicate that probabilistic generative models can achieve competitive multi-label classification performance compared to discriminative methods, and have advantages for datasets with many labels and skewed label frequencies.

A major inference task in Bayesian networks is explaining why some variables are observed in their particular states using a set of target variables. Existing methods for solving this problem often generate explanations that are either too simple (underspecified) or too complex (overspecified). In this paper, we introduce a method called Most Relevant Explanation (MRE) which finds a partial instantiation of the target variables that maximizes the generalized Bayes factor (GBF) as the best explanation for the given evidence. Our study shows that GBF has several theoretical properties that enable MRE to automatically identify the most relevant target variables in forming its explanation. In particular, conditional Bayes factor (CBF), defined as the GBF of a new explanation conditioned on an existing explanation, provides a soft measure on the degree of relevance of the variables in the new explanation in explaining the evidence given the existing explanation. As a result, MRE is able to automatically prune less relevant variables from its explanation. We also show that CBF is able to capture well the explaining-away phenomenon that is often represented in Bayesian networks. Moreover, we define two dominance relations between the candidate solutions and use the relations to generalize MRE to find a set of top explanations that is both diverse and representative. Case studies on several benchmark diagnostic Bayesian networks show that MRE is often able to find explanatory hypotheses that are not only precise but also concise.

We reminisce and discuss applications of algorithmic probability to a wide range of problems in artificial intelligence, philosophy and technological society. We propose that Solomonoff has effectively axiomatized the field of artificial intelligence, therefore establishing it as a rigorous scientific discipline. We also relate to our own work in incremental machine learning and philosophy of complexity.

International networks, although highly efficient, may produce surprising threshold effects that shift costs to geographically distant locations. International utility, transportation, and information networks facilitate the efficient flow of information, energy, goods and people. These networks exhibit a scale-free network structure with a few large “hubs”. Yet their efficiency belies their lack of robustness. Because such networks transcend national boundaries, furthermore, disruptions to the network in one geographic region may have profound economic and national security costs for countries in another region. To illustrate how complex networks may transmit costs among countries, this paper builds an agent-based model (ABM) of the international air transportation system. The ABM employs a genetic algorithm to identify “small” disruptions that produce cascading network failures. The study makes two contributions. First, it demonstrates how some complex networks evolve into network structures that trade off robustness for efficiency. Second, it illustrates how researchers can combine agent-based modeling, evolutionary computation, and network analysis to simulate differing failure modes for global networks. This convergence of simulation methodologies characterizes the emerging field of computational social science.

I ask why humans are smarter than other primates, and I hypothesize that an important part of the answer lies in what I call the Strong Story Hypothesis, which holds that story telling and understanding have a central role in human intelligence. Next, I introduce another hypothesis, the Driven Perception Hypothesis, which holds that we derive much of our commonsense, including the commonsense required in story understanding, by deploying our perceptual apparatus on real and imagined events. Then, after discussing methodology, I describe the representations and methods embodied in the Genesis system, a story-understanding system that analyzes stories ranging from precis of Shakespeare's plots to descriptions of conflicts in cyberspace. The Genesis system works with short story summaries, provided in English, together with low-level commonsense rules and higher-level reflection patterns, likewise expressed in English. Using only a small collection of commonsense rules and reflection patterns, Genesis demonstrates several story-understanding capabilities, such as determining that both Macbeth and the 2007 Russia-Estonia Cyberwar involve revenge, even though neither the word revenge nor any of its synonyms are mentioned. Finally, I describe Rao's Visio-Spatial Reasoning System, a system that recognizes activities such as approaching, jumping, and giving, and answers commonsense questions posed by Genesis.

One approach to bridging the historic divide between "symbolic" and "subsymbolic" AI is to incorporate a subsymbolic system and a symbolic system into a synergetic integrative cognitive architecture. Here we consider various issues related to incorporating (subsymbolic) compositional spatiotemporal deep learning networks (CSDLNs, a term introduced to denote the category including HTM, DeSTIN and other similar systems) into an integrative cognitive architecture including symbolic aspects. The core conclusion is that for such integration to be meaningful, it must involve dynamic and adaptive linkage and conversion between CSDLN attractors spanning sensory, motor and goal hierarchies, and analogous representations in the remainder of the integrative architecture. We suggest the mechanism of "semantic CSDLNs", which maintain the general structure of CSDLNs but contain more abstract patterns, similar to those represented in more explicitly symbolic AI systems. This notion is made concrete by describing a planned integration of the DeSTIN CSDLN into the OpenCog integrative cognitive system (which includes a probabilistic-logical symbolic component).