In this tutorial article, we aim to provide the reader with the conceptual tools needed to get started on research on offline reinforcement learning algorithms: reinforcement learning algorithms that utilize previously collected data, without additional online data collection. Offline reinforcement learning algorithms hold tremendous promise for making it possible to turn large datasets into powerful decision making engines. Effective offline reinforcement learning methods would be able to extract policies with the maximum possible utility out of the available data, thereby allowing automation of a wide range of decision-making domains, from healthcare and education to robotics. However, the limitations of current algorithms make this difficult. We will aim to provide the reader with an understanding of these challenges, particularly in the context of modern deep reinforcement learning methods, and describe some potential solutions that have been explored in recent work to mitigate these challenges, along with recent applications, and a discussion of perspectives on open problems in the field.

There is one truth discovered by every data analyst: datasets are not always available. Most of the times, just to find the specific chunks of data we are searching for we need to scavenge the internet for non existing links, obsolete and badly structured datasets. Sometimes, the data cannot even be found. One issue that you might have encountered already, is that you found the information you were searching for, but not in the form of a dataset. Perhaps, summarized on a graph in a research paper, but not in the form of a downloadable dataset.

The weighted sum method is a simple and widely used technique that scalarizes multiple conflicting objectives into a single objective function. It suffers from the problem of determining the appropriate weights corresponding to the objectives. This paper proposes a novel Hierarchical Bayesian model based on Multinomial distribution and Dirichlet prior to refine the weights for solving such multi-objective route optimization problems. The model and methodologies revolve around data obtained from a small scale pilot survey. The method aims at improving the existing methods of weight determination in the field of Intelligent Transport Systems as data driven choice of weights through appropriate probabilistic modelling ensures, on an average, much reliable results than non-probabilistic techniques. Application of this model and methodologies to simulated as well as real data sets revealed quite encouraging performances with respect to stabilizing the estimates of weights.

ESF thus beckons the valuation of ecosystem services (VES) as a means to signalling nature's contribution to the (re)production of value (Barbier et al., 2009; Villa et al., 2009; Fisher et al., 2010; Gómez-Baggethun et al., 2016); for value is the central category of modern capitalist societies, and the valorisation of value -- i.e., economic growth sublimated into economic development -- their driving force (see, e.g., Mankiw (2016) and Holden et al. (2017)). VES is, in this sense, inscribed in an interpretive approach to modern capitalist praxis, not only invoking assumptions that are instrumentally validated in a retroactive manner, but also taking for granted precisely those historical and material conditions which VES is meant to interpret and, in doing so, reproduce. Overlooking the historical basis of ESF and VES has important practical consequences. When VES practitioners elicit value, a moment or specific field of the social praxis embodied in the valorisation of value is inaugurated, allowing value to mediate other social constructs built around the idea of nature. Since the patterns of actions that make up the capitalist social praxis are presupposed within this new ambit, value takes on a transhistorical quality that justifies its allencompassing and unreflective usage (see, e.g., Badura et al. (2016) and Gómez-Baggethun and Martín-López (2015)).

The economic and banking importance of the small and medium enterprise (SME) sector is well recognized in contemporary society. Business credit loans are very important for the operation of SMEs, and the revenue is a key indicator of credit limit management. Therefore, it is very beneficial to construct a reliable revenue forecasting model. If the uncertainty of an enterprise's revenue forecasting can be estimated, a more proper credit limit can be granted. Natural gradient boosting approach, which estimates the uncertainty of prediction by a multi-parameter boosting algorithm based on the natural gradient. However, its original implementation is not easy to scale into big data scenarios, and computationally expensive compared to state-of-the-art tree-based models (such as XGBoost). In this paper, we propose a Scalable Natural Gradient Boosting Machines that is simple to implement, readily parallelizable, interpretable and yields high-quality predictive uncertainty estimates. According to the characteristics of revenue distribution, we derive an uncertainty quantification function. We demonstrate that our method can distinguish between samples that are accurate and inaccurate on revenue forecasting of SMEs. What's more, interpretability can be naturally obtained from the model, satisfying the financial needs.

Bayes' rule: (or Bayes' theorem) of one probability theory's most important rules, describing the probability of an event, based on prior knowledge of conditions that might be related:

We would like to begin this rejoinder with expressing our sincere gratitude to all of the discussants for their interesting and thought-provoking comments and remarks. We also feel heartily thankful to the editorial board of Bayesian Analysis for giving us the opportunity to publish our paper entitled "A novel algorithmic approach to Bayesian logic regression" (Hubin et al., 2020a) as a discussion article. Logic regression is a tool to model nonlinear relationships between binary covariates and some response variable by constructing predictors as Boolean combinations. The number of possible logic expressions grows exponentially with the number of binary variables involved, making the model search significantly harder with the increasing complexity of Boolean combinations. Due to Boolean equivalence, it is in fact almost impossible to specify the full model space a priori even for a relatively small number of covariates.

Most classification models work by first predicting a posterior probability distribution over all classes and then selecting that class with the largest estimated probability. In many settings however, the quality of posterior probability itself (e.g., 65% chance having diabetes), gives more reliable information than the final predicted class alone. When these methods are shown to be poorly calibrated, most fixes to date have relied on posterior calibration, which rescales the predicted probabilities but often has little impact on final classifications. Here we propose an end-to-end training procedure called posterior calibrated (PosCal) training that directly optimizes the objective while minimizing the difference between the predicted and empirical posterior probabilities.We show that PosCal not only helps reduce the calibration error but also improve task performance by penalizing drops in performance of both objectives. Our PosCal achieves about 2.5% of task performance gain and 16.1% of calibration error reduction on GLUE (Wang et al., 2018) compared to the baseline. We achieved the comparable task performance with 13.2% calibration error reduction on xSLUE (Kang and Hovy, 2019), but not outperforming the two-stage calibration baseline. PosCal training can be easily extendable to any types of classification tasks as a form of regularization term. Also, PosCal has the advantage that it incrementally tracks needed statistics for the calibration objective during the training process, making efficient use of large training sets.

Neural networks are known to suffer from catastrophic forgetting when trained on sequential datasets. While there have been numerous attempts to solve this problem for large-scale supervised classification, little has been done to overcome catastrophic forgetting for few-shot classification problems. We demonstrate that the popular gradient-based few-shot meta-learning algorithm Model-Agnostic Meta-Learning (MAML) indeed suffers from catastrophic forgetting and introduce a Bayesian online meta-learning framework that tackles this problem. Our framework incorporates MAML into a Bayesian online learning algorithm with Laplace approximation. This framework enables few-shot classification on a range of sequentially arriving datasets with a single meta-learned model. The experimental evaluations demonstrate that our framework can effectively prevent forgetting in various few-shot classification settings compared to applying MAML sequentially.

Learning a good representation is an essential component for deep reinforcement learning (RL). Representation learning is especially important in multitask and partially observable settings where building a representation of the unknown environment is crucial to solve the tasks. Here we introduce Prediction of Bootstrap Latents (PBL), a simple and flexible self-supervised representation learning algorithm for multitask deep RL. PBL builds on multistep predictive representations of future observations, and focuses on capturing structured information about environment dynamics. Specifically, PBL trains its representation by predicting latent embeddings of future observations. These latent embeddings are themselves trained to be predictive of the aforementioned representations. These predictions form a bootstrapping effect, allowing the agent to learn more about the key aspects of the environment dynamics. In addition, by defining prediction tasks completely in latent space, PBL provides the flexibility of using multimodal observations involving pixel images, language instructions, rewards and more. We show in our experiments that PBL delivers across-the-board improved performance over state of the art deep RL agents in the DMLab-30 and Atari-57 multitask setting.