Reinforcement learning algorithms typically require too many `trial-and-error' experiences before reaching a desirable behaviour. A considerable amount of ongoing research is focused on speeding up this learning process by using external knowledge. We contribute in several ways, proposing novel approaches to transfer learning and learning from demonstration, as well as an ensemble approach to combine knowledge from various sources.

Social media platforms, such as Twitter, offer a rich source of real-time information about real-world events, particularly during mass emergencies. Sifting valuable information from social media provides useful insight into time-critical situations for emergency officers to understand the impact of hazards and act on emergency responses in a timely manner. This work focuses on analyzing Twitter messages generated during natural disasters, and shows how natural language processing and data mining techniques can be utilized to extract situation awareness information from Twitter. We present key relevant approaches that we have investigated including burst detection, tweet filtering and classification, online clustering, and geotagging.

Decentralized partially observable Markov decision processes (Dec-POMDPs) provide a general model for decision-making under uncertainty in cooperative decentralized settings, but are difficult to solve optimally (NEXP-Complete). As a new way of solving these problems, we recently introduced a method for transforming a Dec-POMDP into a continuous-state deterministic MDP with a piecewise-linear and convex value function. This new Dec-POMDP formulation, which we call an occupancy MDP, allows powerful POMDP and continuous-state MDP methods to be used for the first time. However, scalability remains limited when the number of agents or problem variables becomes large. In this paper, we show that, under certain separability conditions of the optimal value function, the scalability of this approach can increase considerably. This separability is present when there is locality of interaction between agents, which can be exploited to improve performance. Unlike most previous methods, the novel continuous-state MDP algorithm retains optimality and convergence guarantees. Results show that the extension using separability can scale to a large number of agents and domain variables while maintaining optimality.

Earth observation from space allows us to better understand natural phenomenas such as marine currents, to prevent or follow natural disasters, to follow climate evolution and many other things. To achieve that, there are a great number of artificial satellites orbiting Earth, equipped with high-resolution optical instruments and communicating with a network of ground stations. A satellite is said to be agile when it is able to move quickly around its gravity center along its three axes while moving along its orbit, thanks to gyroscopic actuators. It is equipped with a body-mounted optical instrument. To observe a ground area with the instrument, the satellite must be pointed to it. In practice, users submit observation requests to a mission center, which builds activity plans which are sent to the satellites. These plans contain several types of actions such as orbital maneuvers, acquisition realisations and acquisition downloads towards ground stations. Many techniques are used to synthesize such activity plans. Until now, plans are computed offline on the ground and converted into telecommands that the satellite executes strictly, without any flexibility. However, the satellite evolves in a dynamic environment. Unexpected events occur, such as meteorological changes or new urgent observation requests, that the system must handle. Moreover, resource consumption is not always well known. Until now, to ensure that plans will be executable on board with these uncertainties, they are built with worst-case hypothesis on resources consumption. The objective of this work is to give more autonomy to the satellite without compromising the predictability that is needed for some activities. On the ground, we have high computing power and high uncertainty, while on board we have very low computing power and low uncertainty. The main idea is to share decision-making between ground and board to take advantage of the high computing power on the ground and of the low uncertainty on board. First we apply this idea to download scheduling which consists in scheduling file downloads during ground station visibility windows. Second, we apply this idea to observation planning.

The latest research on prediction of the outcome of elections using Twitter data, the election tweets labeling area has hardly been explored. Therefore, the authors of this paper propose to develop a semi-automated model for labeling political tweets. The expected result of this study is to contribute to enhance the quality of the choice of messages used in the labeling process by reducing the time selection of messages and the efficiency of classifying the messages and, thus, to increase the accuracy of the models using this approach. The proposed method could label 2200 messages from the analysis of only 60 messages by 20 users. The first results obtained by the method were higher than the process carried out manually by humans.

Rosetta is a European Space Agency (ESA) cornerstone mission that entered orbit around the comet 67P/Churyumov-Gerasimenko in August 2014 and will escort the comet for a 1.5 year nominal mission offering the most detailed study of a comet ever undertaken by humankind. The Rosetta orbiter has 11 scientific instruments (4 remote sensing) and the Philae lander to make complementary measurements of the comet nucleus, coma (gas and dust), and surrounding environment. The ESA Rosetta Science Ground Segment has developed a science scheduling system that includes an automated scheduling capability to assist in developing science plans for the Rosetta Orbiter. While automated scheduling is a small portion of the overall Science Ground Segment (SGS) as well as the overall scheduling system, this paper focuses on the automated and semi-automated scheduling software (called ASPEN-RSSC) and how this software is used.

We introduce a novel kernel that upgrades the Weisfeiler-Lehman and other graph kernels to effectively exploit high-dimensional and continuous vertex attributes. Graphs are first decomposed into subgraphs. Vertices of the subgraphs are then compared by a kernel that combines the similarity of their labels and the similarity of their structural role, using a suitable vertex invariant. By changing this invariant we obtain a family of graph kernels which includes generalizations of Weisfeiler-Lehman, NSPDK, and propagation kernels. We demonstrate empirically that these kernels obtain state-of-the-art results on relational data sets.

Nowadays web users actively generate content on different social media platforms. The large number of users requiring personalized services creates a unique opportunity for researchers to explore user modelling. Substantial research has been done by utilizing user generated content to model users by applying different classification or regression techniques. These techniques are powerful types of machine learning approaches, however they only partially model social media users. In this work, we introduce a new statistical relational learning (SRL) framework suitable for this purpose, which we call PSL Q . PSL Q is the first SRL framework that supports reasoning with soft quantifiers, such as “most” and “a few”. Indeed, in models for social media it is common to assume that friends are influenced by each other’s behavior, beliefs, and preferences. Thus, having a trait only becomes probable once most or some of one’s friends have that trait. Expressing this dependency requires a soft quantifier, which can be modeled with PSL^Q. Our experimental results for link prediction in social trust networks demonstrate that the use of soft quantifiers not only allows for a natural and intuitive formulation of domain knowledge, but also improves the accuracy of inferred results.

In this work, we deal with a relatively new statistical tool in machine learning: the estimation of the ratio of two probability densities, or density ratio estimation for short. As a side piece of research that gained its own traction, we also tackle the task of parameter selection in learning algorithms based on kernel methods.

In decision-theoretic planning problems, such as (partially observable) Markov decision problems or coordination graphs, agents typically aim to optimize a scalar value function. However, in many real-world problems agents are faced with multiple possibly conflicting objectives. In such multi-objective problems, the value is a vector rather than a scalar, and we need methods that compute a coverage set, i.e., a set of solutions optimal for all possible trade-offs between the objectives. In this project propose new multi-objective planning methods that compute the so-called convex coverage set (CCS): the coverage set for when policies can be stochastic, or the preferences are linear. We show that the CCS has favorable mathematical properties, and is typically much easier to compute that the Pareto front, which is often axiomatically assumed as the solution set for multi-objective decision problems.