Appropriately designing sampling policies is highly important for obtaining better control policies in reinforcement learning. In this paper, we first show that the least-squares policy iteration (LSPI) framework allows us to employ statistical active learning methods for linear regression. Then we propose a design method of good sampling policies for efficient exploration, which is particularly useful when the sampling cost of immediate rewards is high. We demonstrate the usefulness of the proposed method, named active policy iteration (API), through simulations with a batting robot.

Random Forests have been shown to perform very well in propositional learning.  FORF is an upgrade of Random Forests for relational data. In this paper we investigate shortcomings of FORF and propose an alternative algorithm, RF, for generating Random Forests over relational data. RF employs randomly generated relational rules as fully self-contained Boolean tests inside each node in a tree and thus can be viewed as an instance of dynamic propositionalization.  The implementation of RF allows for the simultaneous or parallel growth of all the branches of all the trees in the ensemble in an efficient shared, but still single-threaded way.  Experiments favorably compare RF to both FORF and the combination of static propositionalization together with standard Random Forests. Various strategies for tree initialization and splitting of nodes, as well as resulting ensemble size, diversity, and computational complexity of RF are also investigated.

Qualitative constraint calculi are representation formalisms that allow for efficient reasoning about spatial and temporal information. Many of the calculi discussed in the field of Qualitative Spatial and Temporal Reasoning can be defined as combinations of other, simpler and more compact formalisms. On the other hand, existing calculi can be combined to a new formalism in which one can represent, and reason about, different aspects of a domain at the same time. For example, Gerevini and Renz presented a loose combination of the region connection calculus RCC-8 and the point algebra: the resulting formalism integrates topological and qualitative size relations between spatially extended objects. In this paper we compare the approach by Gerevini and Renz to a method that generates a new qualitative calculus by exploiting the semantic interdependencies between the component calculi. We will compare these two methods and analyze some formal relationships between a combined calculus and its components. The paper is completed by an empirical case study in which the reasoning performance of the suggested methods is compared on random test instances.

A fundamental task for reasoning with preferences is the following: given inputpreference information from a user, and outcomes α and β, should we infer that the user will prefer α to β? For CP-nets and related comparative preference formalisms, inferring a preference of α over β using the standard definition of derived preference appears to be extremely hard, and has been proved to be PSPACE-complete in general for CP-nets. Such inference is also rather conservative, only making the assumption of transitivity. This paper defines a less conservative approach to inference which can be applied for very general forms of input. It is shown to be efficient for expressive comparative preference languages, allowing comparisons between arbitrary partial tuples (including complete assignments), and with the preferences being ceteris paribus or not.

Modal correspondence theory is a powerful and effective way to guarantee that adding specific syntactic axioms to a modal logic is mirrored by requiring 'corresponding' properties of the underlying Kripke models. However, such axioms not only  quantify over all formulas, but they are also global  in the sense that the corresponding semantic property is assumed to hold for all states. However, in  for instance epistemic logic one would like to have  the flexibility to say that certain properties (like "agent b knows at least what agent a knows") are  true locally in a specific state, but not necessarily  globally, in all states. This would enable one to  say "currently, b knows at least what a knows, but  this is not common knowledge," or ". . . but this is  not always true," or ". . . but this could be changed  by action α." We offer a logic for "knowing at least  as," where the (global) axiom scheme Ka ϕ → Kb ϕ  is replaced by a (local) inference rule. We give  a complete modal system, and discuss some consequences of the axiom in an epistemic setting.  Our completeness proof also suggests how achieving such local properties can be generalized to other  axioms schemes and modal logics. 

We review and extend earlier work on the logic CFD, a description logic that allows terminological cycles with universal restrictions over functional roles. In particular, we consider the problem of reasoning about concept subsumption and the problem of computing certain answers for a family of attribute-connected conjunctive queries, showing that both problems are in PTIME. We then consider the effect on the complexity of these problems after adding a concept constructor that expresses concept union, or after adding a concept constructor for the bottom class. Finally, we show that adding both constructors makes both problems EXPTIME-complete.

This paper considers the problem of composing or scheduling several (non-deterministic) behaviors so as to conform to a specified target behavior as well as satisfying constraints imposed by the environment in which the behaviors are to be performed. This problem has already been considered by several works in the literature and applied to areas such as web service composition, the composition of robot behaviors and co-ordination of distributed devices. We develop a sound and complete algorithm for determining such a composition which has a number of significant advantages over previous proposals: a) our algorithm is different from previous proposals which resort to dynamic logic or simulation relations, b) we realized an implementation in Java as opposed to other approaches for which there are no known implementations, c) our algorithm determines all possible schedulers at once, and d) we can use our framework to define a notion of approximation when the target behavior cannot be realized.

This is also a key issue in formalizing deals with incomplete information, preferences, negotiation, which seems to prefer argumentationbased and changing goals. We assume that each negotiation [Rahwan et al., 2003]. Recent proposals agent is equipped with a knowledge base for negotiation on formalizing negotiation (see, e.g., [Amgoud et al., 2006; which consists of a CRprogram, a set of possible Kakas and Moraitis, 2006; Rahwan et al., 2003]) seem to be assumptions, and a set of ordered goals.

We consider query answering on Description Logic (DL) ontologies with DBoxes, where a DBox is a set of assertions on individuals involving atomic concepts and roles called DBox predicates. The extension of a DBox predicate is exactly defined in every interpretation by the contents of the DBox, i.e., a DBox faithfully represents a database whose table names are the DBox predicates and the tuples are the DBox assertions. Our goals are (i) to find out whether the answers to a given query are solely determined by the DBox predicates and, if so, (ii) to find a rewriting of the query in terms of them. The resulting query can then be efficiently evaluated using standard database technology. We have that (i) can be reduced to entailment checking and (ii) can be reduced to finding an interpolant. We present a procedure for computing interpolants in the DL ALC with general TBoxes. We extend the procedure with standard tableau optimisations, and we discuss abduction as a technique for amending ontologies to gain definability of queries of interest.

It has been recognised that the expressivity of description logics benefits from the introduction of non-standard modal operators beyond existential and number restrictions. Such operators support notions such as uncertainty, defaults, agency, obligation, or evidence, whose semantics often lies outside the realm of relational structures. Coalgebraic hybrid logic serves as a unified setting for logics that combine non-standard modal operators and nominals, which allow reasoning about individuals. In this framework, we prove a generic EXPTIME upper bound for concept satisfiability over general TBoxes, which instantiates to novel upper bounds for many individual logics including probabilistic logic with nominals.