Contextual bandits are online learners that, given an input, select an arm and receive a reward for that arm. They use the reward as a learning signal and aim to maximize the total reward over the inputs. Contextual bandits are commonly used to solve recommendation or ranking problems. This paper considers a learning setting in which multiple parties aim to train a contextual bandit together in a private way: the parties aim to maximize the total reward but do not want to share any of the relevant information they possess with the other parties. Specifically, multiple parties have access to (different) features that may benefit the learner but that cannot be shared with other parties. One of the parties pulls the arm but other parties may not learn which arm was pulled. One party receives the reward but the other parties may not learn the reward value. This paper develops a privacy-preserving contextual bandit algorithm that combines secure multi-party computation with a differential private mechanism based on epsilon-greedy exploration in contextual bandits.

Magnetoencephalography and electroencephalography (M/EEG) are non-invasive modalities that measure the weak electromagnetic fields generated by neural activity. Estimating the location and magnitude of the current sources that generated these electromagnetic fields is a challenging ill-posed regression problem known as \emph{source imaging}. When considering a group study, a common approach consists in carrying out the regression tasks independently for each subject. An alternative is to jointly localize sources for all subjects taken together, while enforcing some similarity between them. By pooling all measurements in a single multi-task regression, one makes the problem better posed, offering the ability to identify more sources and with greater precision. The Minimum Wasserstein Estimates (MWE) promotes focal activations that do not perfectly overlap for all subjects, thanks to a regularizer based on Optimal Transport (OT) metrics. MWE promotes spatial proximity on the cortical mantel while coping with the varying noise levels across subjects. On realistic simulations, MWE decreases the localization error by up to 4 mm per source compared to individual solutions. Experiments on the Cam-CAN dataset show a considerable improvement in spatial specificity in population imaging. Our analysis of a multimodal dataset shows how multi-subject source localization closes the gap between MEG and fMRI for brain mapping.

We compare the model-free reinforcement learning with the model-based approaches through the lens of the expressive power of neural networks for policies, $Q$-functions, and dynamics. We show, theoretically and empirically, that even for one-dimensional continuous state space, there are many MDPs whose optimal $Q$-functions and policies are much more complex than the dynamics. We hypothesize many real-world MDPs also have a similar property. For these MDPs, model-based planning is a favorable algorithm, because the resulting policies can approximate the optimal policy significantly better than a neural network parameterization can, and model-free or model-based policy optimization rely on policy parameterization. Motivated by the theory, we apply a simple multi-step model-based bootstrapping planner (BOOTS) to bootstrap a weak $Q$-function into a stronger policy. Empirical results show that applying BOOTS on top of model-based or model-free policy optimization algorithms at the test time improves the performance on MuJoCo benchmark tasks.

How are you using Watson at Nedbank? Traditionally, Nedbank has reduced rates of online fraud by using rule-based decision systems. Every time a fraud was committed, new rules were added. Unfortunately, this created a large catch-all problem where responders would see many false alarms, which would encumber resources and divert attention away from actual fraud. By using machine learning, developed within the Watson Local development environment, we were able to drop the false positive rate from 60–80% to below 35%. Watson helped with easy collaboration for model development and simplified the deployment process by letting us build a production-ready API with ease.

Collecting large-scale data with clean labels for supervised training of neural networks is practically challenging. Although noisy labels are usually cheap to acquire, existing methods suffer severely for training datasets with high noise ratios, making high-cost human labeling a necessity. Here we present a method to train neural networks in a way that is almost invulnerable to severe label noise by utilizing a tiny trusted set. Our method, named IEG, is based on three key insights: (i) Isolation of noisy labels, (ii) Escalation of useful supervision from mislabeled data, and (iii) Guidance from small trusted data. On CIFAR100 with a 40% uniform noise ratio and 10 trusted labeled data per class, our method achieves 80. 2 0.3% classification accuracy, only 1.4% higher error than a neural network trained without label noise. Moreover, increasing the noise ratio to 80%, our method still achieves a high accuracy of 75 .5 Training deep neural networks usually requires large-scale labeled data. However, the process of data labelling by humans is challenging and expensive in practice, especially in domains where expert annotators are needed such as medical imaging. A great number of methods have been proposed to train neural networks from datasets with noisy labels due to cheap acquisition (e.g.

Predicting discomfort glare in open-plan offices is a challenging problem since most of available glare metrics are developed for cellular offices which are typically daylight dominated. The problem with open-plan offices is that they are mainly dependent on electric lighting rather than daylight even when they have a fully glazed facade. In addition, the contrast between bright windows and the buildings interior can be problematic and may cause discomfort glare to the building occupants. These problems can affect occupant productivity and wellbeing. Thus, it is important to develop a predictive model to avoid discomfort glare when designing open plan offices. To the best of our knowledge, we are the first to adopt Machine Learning (ML) models to predict discomfort glare. In order to develop new glare predictive models for these types of offices, Post-Occupancy Evaluation (POE) and High Dynamic Range (HDR) images were collected from 80 occupants (n=80) in four different open-plan offices. Consequently, various multi-region luminance values, luminance and glare indices were calculated and used as input features to train ML models. The accuracy of the ML model was compared to the accuracy of 24 indices which were also evaluated using a Receiver Operating Characteristic (ROC) analysis to identify the best cutoff values (thresholds) for each index for open-plan configurations. Results showed that the ML glare model could predict glare in open-plan offices with an accuracy of 83.8% (0.80 true positive rate and 0.86 true negative rate) which outperformed the accuracy of the previously developed glare metrics.

Recently, machine learning (ML) has introduced advanced solutions to many domains. Since ML models provide business advantage to model owners, protecting intellectual property (IP) of ML models has emerged as an important consideration. Confidentiality of ML models can be protected by exposing them to clients only via prediction APIs. However, model extraction attacks can steal the functionality of ML models using the information leaked to clients through the results returned via the API. In this work, we question whether model extraction is a serious threat to complex, real-life ML models. We evaluate the current state-of-the-art model extraction attack (the Knockoff attack) against complex models. We reproduced and confirm the results in the Knockoff attack paper. But we also show that the performance of this attack can be limited by several factors, including ML model architecture and the granularity of API response. Furthermore, we introduce a defense based on distinguishing queries used for Knockoff attack from benign queries. Despite the limitations of the Knockoff attack, we show that a more realistic adversary can effectively steal complex ML models and evade known defenses.

Models for different classification problems can be fitted by trying to maximize or minimize various performance measures. Measurements that address one aspect of a model's performance but not another are important to note so that we can make an informed decision and select the performance measures that best fit our design. ROC AUC is commonly used in many fields as a prominent measure to evaluate classifier performance, and researchers might favor one classifier over another due to a higher AUC. For a refresher on ROC AUC, a clear and concise explanation can be found here. If you are totally unfamiliar with ROC AUC you may find that this post digs into the subject a bit too deep, but I hope you will still find it useful or bookmark it for future reference.

This wiki is about bootstrapping. "Recipe for yogurt: Add yogurt to milk." - Anon. Also see http://bootstrappable.org, which has pointers to a mailing list and IRC channel. Simple explanation: bootstrapping is about building a compiler using tools smaller than itself, as opposed to building a compiler using an already built version of itself. The problem with the second is: Where did that prebuilt binary come from?

Specifically, without knowing the future p -values, the analyst must irrevocably decide at each step whether to reject the null, such that with probability at least 1 α, there are no false rejections in the entire sequence. This paper unifies algorithm design concepts developed for offline FWER control and for online false discovery rate (FDR) control. Though Bonferroni, fallback procedures and Sidak's method can trivially be extended to the online setting, our main contribution is the design of new, adaptive online algorithms that control the FWER and per-family error rate (PFER) when the p -values are independent or locally dependent in time. Our experiments demonstrate substantial gains in power, also formally proved in an idealized Gaussian model. 1 Introduction Online multiple testing refers to the setting in which a potentially infinite stream of hypotheses H 1,H 2,... (respectively p -values P 1,P 2,...) is tested sequentially one at a time. At each step t N, one must decide whether to reject the current null hypothesis H t or not, without knowing the outcomes of all the future tests. Typically, we reject the null hypothesis when P t is smaller than some threshold α t. Let R represent the set of rejected null hypotheses, and H 0 be the unknown set of true null hypotheses; then, V R H 0 is the set of incorrectly rejected null hypotheses, also known as false discoveries. Denoting V V, some common error metrics are the false discovery rate (FDR), family wise error rate (FWER), per-family error rate (PFER) and power which are defined as FDR E null V R 1 null, FWER Pr{ V 1}, PFER E [V ], power E null H c 0 R H c 0 null .