A key task in multi-label classification is modeling the structure between the involved classes. Modeling this structure by probabilistic and interpretable means enables application in a broad variety of tasks such as zero-shot learning or learning from incomplete data. In this paper, we present the probabilistic rule stacking learner (pRSL) which uses probabilistic propositional logic rules and belief propagation to combine the predictions of several underlying classifiers. We derive algorithms for exact and approximate inference and learning, and show that pRSL reaches state-of-the-art performance on various benchmark datasets. In the process, we introduce a novel multicategorical generalization of the noisy-or gate. Additionally, we report simulation results on the quality of loopy belief propagation algorithms for approximate inference in bipartite noisy-or networks.

With the increased interest in machine learning and big data problems, the need for large amounts of labelled data has also grown. However, it is often infeasible to get experts to label all of this data, which leads many practitioners to crowdsourcing solutions. In this paper, we present new techniques to improve the quality of the labels while attempting to reduce the cost. The naive approach to assigning labels is to adopt a majority vote method, however, in the context of data labelling, this is not always ideal as data labellers are not equally reliable. One might, instead, give higher priority to certain labellers through some kind of weighted vote based on past performance. This paper investigates the use of more sophisticated methods, such as Bayesian inference, to measure the performance of the labellers as well as the confidence of each label. The methods we propose follow an iterative improvement algorithm which attempts to use the least amount of workers necessary to achieve the desired confidence in the inferred label. This paper explores simulated binary classification problems with simulated workers and questions to test the proposed methods. Our methods outperform the standard voting methods in both cost and accuracy while maintaining higher reliability when there is disagreement within the crowd.

Santiago, Chile, is a highly segregated city with distinct zones of affluence and deprivation. This setting offers a window on how social factors propel the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) pandemic in an economically vulnerable society with high levels of income inequality. Mena et al. analyzed incidence and mortality attributed to SARS-CoV-2 to understand spatial variations in disease burden. Infection fatality rates were higher in lower-income municipalities because of comorbidities and lack of access to health care. Disparities between municipalities in the quality of their health care delivery system became apparent in testing delays and capacity. These indicators explain a large part of the variation in COVID-19 underreporting and deaths and show that these inequalities disproportionately affected younger people. Science , abg5298, this issue p. [eabg5298][1] ### INTRODUCTION The COVID-19 crisis has exposed major inequalities between communities. Understanding the societal risk factors that make some groups particularly vulnerable is essential to ensure more effective interventions for this and future pandemics. Here, we focus on socioeconomic status as a risk factor. Although it is broadly understood that social and economic inequality has a negative impact on health outcomes, the mechanisms by which socioeconomic status affects disease outcomes remain unclear. These mechanisms can be mediated by a range of systemic structural factors, such as access to health care and economic safety nets. We address this gap by providing an in-depth characterization of disease incidence and mortality and their dependence on demographic and socioeconomic strata in Santiago, a highly segregated city and the capital of Chile. ### RATIONALE Combining publicly available data sources, we conducted a comprehensive analysis of case incidence and mortality during the first wave of the pandemic. We correlated COVID-19 outcomes with behavioral and health care system factors while studying their interaction with age and socioeconomic status. To overcome the intrinsic biases of incomplete case count data, we used detailed mortality data. We developed a parsimonious Gaussian process model to study excess deaths and their uncertainty and reconstructed true incidence from the time series of deaths with a new regularized maximum likelihood deconvolution method. To estimate infection fatality rates by age and socioeconomic status, we implemented a hierarchical Bayesian model that adjusts for reporting biases while accounting for incompleteness in case information. ### RESULTS We find robust associations between COVID-19 outcomes and socioeconomic status, based on health and behavioral indicators. Specifically, we show in lower–socioeconomic status municipalities that testing was almost absent early in the pandemic and that human mobility was not reduced by lockdowns as much as it was in more affluent locations. Test positivity and testing delays were much higher in these locations, indicating an impaired capacity of the health care system to contain the spread of the epidemic. We also find that 73% more deaths than in a normal year were observed between May and July 2020, and municipalities at the lower end of the socioeconomic spectrum were hit the hardest, both in relation to COVID-19–attributed deaths and excess deaths. Finally, the socioeconomic gradient of the infection fatality rate appeared particularly steep for younger age groups, reflecting worse baseline health status and limited access to health care in municipalities with low socioeconomic status. ### CONCLUSION Together, these findings highlight the substantial consequences of socioeconomic and health care disparities in a highly segregated city and provide practical methodological approaches useful for characterizing the COVID-19 burden and mortality in other urban centers based on public data, even if reports are incomplete and biased. ![Figure][2] Effect of socioeconomic inequalities on COVID-19 outcomes. The map on the left shows the municipalities that were included in this study, colored by their socioeconomic status score. For the comparison between COVID-19 deaths and excess deaths (top right), COVID-19–confirmed deaths are shown in light green and COVID-19–attributed deaths in dark green. Excess deaths, shown in gray, correspond to the difference between observed and predicted deaths. Predicted deaths were estimated using a Gaussian process model. The shading indicates 95% credible intervals for the excess deaths. The infection fatality rates (bottom right) were inferred by implementing a hierarchical Bayesian model, with vertical lines representing credible intervals by age and socioeconomic status. The COVID-19 pandemic has affected cities particularly hard. Here, we provide an in-depth characterization of disease incidence and mortality and their dependence on demographic and socioeconomic strata in Santiago, a highly segregated city and the capital of Chile. Our analyses show a strong association between socioeconomic status and both COVID-19 outcomes and public health capacity. People living in municipalities with low socioeconomic status did not reduce their mobility during lockdowns as much as those in more affluent municipalities. Testing volumes may have been insufficient early in the pandemic in those places, and both test positivity rates and testing delays were much higher. We find a strong association between socioeconomic status and mortality, measured by either COVID-19–attributed deaths or excess deaths. Finally, we show that infection fatality rates in young people are higher in low-income municipalities. Together, these results highlight the critical consequences of socioeconomic inequalities on health outcomes. [1]: /lookup/doi/10.1126/science.abg5298 [2]: pending:yes

A challenge that machine learning practitioners in the industry face is the task of selecting the best model to deploy in production. As a model is often an intermediate component of a production system, online controlled experiments such as A/B tests yield the most reliable estimation of the effectiveness of the whole system, but can only compare two or a few models due to budget constraints. We propose an automated online experimentation mechanism that can efficiently perform model selection from a large pool of models with a small number of online experiments. We derive the probability distribution of the metric of interest that contains the model uncertainty from our Bayesian surrogate model trained using historical logs. Our method efficiently identifies the best model by sequentially selecting and deploying a list of models from the candidate set that balance exploration-exploitation. Using simulations based on real data, we demonstrate the effectiveness of our method on two different tasks.

We report on the potential for using algorithms for non-negative matrix factorization (NMF) to improve parameter estimation in topic models. While several papers have studied connections between NMF and topic models, none have suggested leveraging these connections to develop new algorithms for fitting topic models. Importantly, NMF avoids the "sum-to-one" constraints on the topic model parameters, resulting in an optimization problem with simpler structure and more efficient computations. Building on recent advances in optimization algorithms for NMF, we show that first solving the NMF problem then recovering the topic model fit can produce remarkably better fits, and in less time, than standard algorithms for topic models. While we focus primarily on maximum likelihood estimation, we show that this approach also has the potential to improve variational inference for topic models. Our methods are implemented in the R package fastTopics.

We present a Bayesian treatment for deep regression using an Errors-in-Variables model which accounts for the uncertainty associated with the input to the employed neural network. It is shown how the treatment can be combined with already existing approaches for uncertainty quantification that are based on variational inference. Our approach yields a decomposition of the predictive uncertainty into an aleatoric and epistemic part that is more complete and, in many cases, more consistent from a statistical perspective. We illustrate and discuss the approach along various toy and real world examples.

Deep ensembles can be seen as the current state-of-the-art for uncertainty quantification in deep learning. While the approach was originally proposed as an non-Bayesian technique, arguments towards its Bayesian footing have been put forward as well. We show that deep ensembles can be viewed as an approximate Bayesian method by specifying the corresponding assumptions. Our finding leads to an improved approximation which results in an increased epistemic part of the uncertainty. Numerical examples suggest that the improved approximation can lead to more reliable uncertainties. Analytical derivations ensure easy calculation of results.

"The key distinguishing property of a Bayesian approach is marginalization instead of optimization, where we represent solutions given by all settings of parameters weighted by their posterior probabilities, rather than bet everything on a single setting of parameters." The time is ripe to dig into marginalization vs optimization, and broaden our general understanding of the Bayesian approach.

While the choice of prior is one of the most critical parts of the Bayesian inference workflow, recent Bayesian deep learning models have often fallen back on vague priors, such as standard Gaussians. In this review, we highlight the importance of prior choices for Bayesian deep learning and present an overview of different priors that have been proposed for (deep) Gaussian processes, variational autoencoders, and Bayesian neural networks. We also outline different methods of learning priors for these models from data. We hope to motivate practitioners in Bayesian deep learning to think more carefully about the prior specification for their models and to provide them with some inspiration in this regard.

MIT press provides another excellent book in creative commons. I plan to buy it and I recommend you do. This book provides a broad introduction to algorithms for decision making under uncertainty. An agent is an entity that acts based on observations of its environment. The interaction between the agent and the environment follows an observe-act cycle or loop.