Model-based reinforcement learning (RL) methods can be broadly categorized as global model methods, which depend on learning models that provide sensible predictions in a wide range of states, or local model methods, which iteratively refit simple models that are used for policy improvement. While predicting future states that will result from the current actions is difficult, local model methods only attempt to understand system dynamics in the neighborhood of the current policy, making it possible to produce local improvements without ever learning to predict accurately far into the future. The main idea in this paper is that we can learn representations that make it easy to retrospectively infer simple dynamics given the data from the current policy, thus enabling local models to be used for policy learning in complex systems. To that end, we focus on learning representations with probabilistic graphical model (PGM) structure, which allows us to devise an efficient local model method that infers dynamics from real-world rollouts with the PGM as a global prior. We compare our method to other model-based and model-free RL methods on a suite of robotics tasks, including manipulation tasks on a real Sawyer robotic arm directly from camera images. Videos of our results are available at https://sites.google.com/view/solar-iclips

Spatio-temporal problems are ubiquitous and of vital importance in many research fields. Despite the potential already demonstrated by deep learning methods in modeling spatio-temporal data, typical approaches tend to focus solely on conditional expectations of the output variables being modeled. In this paper, we propose a multi-output multi-quantile deep learning approach for jointly modeling several conditional quantiles together with the conditional expectation as a way to provide a more complete "picture" of the predictive density in spatio-temporal problems. Using two large-scale datasets from the transportation domain, we empirically demonstrate that, by approaching the quantile regression problem from a multi-task learning perspective, it is possible to solve the embarrassing quantile crossings problem, while simultaneously significantly outperforming state-of-the-art quantile regression methods. Moreover, we show that jointly modeling the mean and several conditional quantiles not only provides a rich description about the predictive density that can capture heteroscedastic properties at a neglectable computational overhead, but also leads to improved predictions of the conditional expectation due to the extra information and a regularization effect induced by the added quantiles.

With deployments of online monitoring systems in distribution networks, massive amounts of data collected through them contain rich information on the operating status of distribution networks. By leveraging the data, based on bidirectional generative adversarial networks (BiGANs), we propose an unsupervised approach for online distribution reliability assessment. It is capable of discovering the latent structure and automatically learning the most representative features of the spatio-temporal data in distribution networks in an adversarial way and it does not rely on any assumptions of the input data. Based on the extracted features, a statistical magnitude for them is calculated to indicate the data behavior. Furthermore, distribution reliability states are divided into different levels and we combine them with the calculated confidence level $1-\alpha$, during which clear criteria is defined empirically. Case studies on both synthetic data and real-world online monitoring data show that our proposed approach is feasible for the assessment of distribution operation reliability and outperforms other existed techniques.

Smart devices of everyday use (such as smartphones and wearables) are increasingly integrated with sensors that provide immense amounts of information about a person's daily life such as behavior and context. The automatic and unobtrusive sensing of behavioral context can help develop solutions for assisted living, fitness tracking, sleep monitoring, and several other fields. Towards addressing this issue, we raise the question: can a machine learn to recognize a diverse set of contexts and activities in a real-life through joint learning from raw multi-modal signals (e.g. accelerometer, gyroscope and audio etc.)? In this paper, we propose a multi-stream temporal convolutional network to address the problem of multi-label behavioral context recognition. A four-stream network architecture handles learning from each modality with a contextualization module which incorporates extracted representations to infer a user's context. Our empirical evaluation suggests that a deep convolutional network trained end-to-end achieves an optimal recognition rate. Furthermore, the presented architecture can be extended to include similar sensors for performance improvements and handles missing modalities through multi-task learning without any manual feature engineering on highly imbalanced and sparsely labeled dataset.

Exponential and concentration inequalities are corner stones of machine learning theory. The first distribution-free bounds on the Empirical Risk Minimiser (ERM), proven by Vapnik and Cervnonenkis in the early 70s, are based on Hoeffding's inequality, see Vapnik (1998). Model selection techniques rely heavily on concentration inequalities (Massart (2007)). We defer the reader to Boucheron et al. (2013) for an overview on concentration inequalities. However, all the results in these references are in the case of i.i.d random variables. Many extensions of Hoeffding and Bernstein's inequalities were proposed for dependent observations: see Catoni (2003); Bertail and Clémençon (2010); Joulin and Ollivier (2010); Dedecker and Fan (2015); Fan et al. (2018) under

In order to exploit quantum advantages, quantum algorithms are indispensable for operating machine learning with quantum computers. We here propose an intriguing hybrid approach of quantum information processing for quantum linear regression, which utilizes both discrete and continuous quantum variables, in contrast to existing wisdoms based solely upon discrete qubits. In our framework, data information is encoded in a qubit system, while information processing is tackled using auxiliary continuous qumodes via qubit-qumode interactions. Moreover, it is also elaborated that finite squeezing is quite helpful for efficiently running the quantum algorithms in realistic setup. Comparing with an all-qubit approach, the present hybrid approach is more efficient and feasible for implementing quantum algorithms, still retaining exponential quantum speed-up.

Latest deep learning methods for object detection provide remarkable performance, but have limits when used in robotic applications. One of the most relevant issues is the long training time, which is due to the large size and imbalance of the associated training sets, characterized by few positive and a large number of negative examples (i.e. background). Proposed approaches are based on end-to-end learning by back-propagation [22] or kernel methods trained with Hard Negatives Mining on top of deep features [8]. These solutions are effective, but prohibitively slow for on-line applications. In this paper we propose a novel pipeline for object detection that overcomes this problem and provides comparable performance, with a 60x training speedup. Our pipeline combines (i) the Region Proposal Network and the deep feature extractor from [22] to efficiently select candidate RoIs and encode them into powerful representations, with (ii) the FALKON [23] algorithm, a novel kernel-based method that allows fast training on large scale problems (millions of points). We address the size and imbalance of training data by exploiting the stochastic subsampling intrinsic into the method and a novel, fast, bootstrapping approach. We assess the effectiveness of the approach on a standard Computer Vision dataset (PASCAL VOC 2007 [5]) and demonstrate its applicability to a real robotic scenario with the iCubWorld Transformations [18] dataset.

A number of techniques have been proposed for aspect discovery using part of speech tagging (Hu and Liu, 2004), syntactic parsing (Lu et al., 2009), clustering (Mei et al., 2007; Titov and McDonald, 2008b), data mining (Ku et al., 2006), and information extraction (Popescu and Etzioni, 2005). Various lexicon and rule-based methods (Hu and Liu, 2004; Ku et al., 2006; Blair-Goldensohn et al., 2008) have been adopted for sentiment prediction together with a few learning approaches (Lu et al., 2009; Pappas and Popescu-Belis, 2017; Angelidis and Lapata, 2018). As for the summaries, a common format involves a list of aspects and the number of positive and negative opinions for each (Hu and Liu, 2004). While this format gives an overall idea of people's opinion, reading the actual text might be necessary to gain a better understanding of specific details. Textual summaries are created following mostly extractive methods (but see Ganesan et al. 2010 for an abstractive approach), and various formats ranging from lists of words (Popescu and Etzioni, 2005), to phrases (Lu et al., 2009), and sentences (Mei et al., 2007; Blair-Goldensohn et al., 2008; Lerman et al., 2009; Wang and Ling, 2016). In this paper, we present a neural framework for opinion extraction from product reviews. We follow the standard architecture for aspect-based summarization, while taking advantage of the success of neural network models in learning continuous features without recourse to preprocessing tools or linguistic annotations.

Recent machine learning strategies for segmentation tasks have shown great ability when trained on large pixel-wise annotated image datasets. It remains a major challenge however to aggregate such datasets, as the time and monetary cost associated with collecting extensive annotations is extremely high. This is particularly the case for generating precise pixel-wise annotations in video and volumetric image data. To this end, this work presents a novel framework to produce pixel-wise segmentations using minimal supervision. Our method relies on 2D point supervision, whereby a single 2D location within an object of interest is provided on each image of the data. Our method then estimates the object appearance in a semi-supervised fashion by learning object-image-specific features and by using these in a semi-supervised learning framework. Our object model is then used in a graph-based optimization problem that takes into account all provided locations and the image data in order to infer the complete pixel-wise segmentation. In practice, we solve this optimally as a tracking problem using a K-shortest path approach. Both the object model and segmentation are then refined iteratively to further improve the final segmentation. We show that by collecting 2D locations using a gaze tracker, our approach can provide state-of-the-art segmentations on a range of objects and image modalities (video and 3D volumes), and that these can then be used to train supervised machine learning classifiers.

As machine learning algorithms are increasing used in security-critical applications, there is a growing need to design them with active adversaries in mind. A class of adversarial attacks on machine learning that have received much attention is data poisoning attacks [21, 20, 19, 4, 7, 11]. Here, an adversary is aware of the learner's training data and algorithm, and has the power to alter a small fraction of the training data in order to make the trained classifier satisfy certain objectives. For example, a sabotage adversary may try to degrade the overall accuracy of the trained classifier as part of an industrial sabotage campaign, or a profit-oriented adversary may try to poison the training data so that the resulting model favors it - say, by recommending the its products over others. While there has been a long line of prior work on data poisoning [21, 20, 19, 4, 7, 11, 3, 8], most of it has focussed in the offline setting, where a classifier or some other model is trained on a fixed input.