Many scientific questions require estimating the effects of continuous treatments. Outcome modeling and weighted regression based on the generalized propensity score are the most commonly used methods to evaluate continuous effects. However, these techniques may be sensitive to model misspecification, extreme weights or both. In this paper, we propose Kernel Optimal Orthogonality Weighting (KOOW), a convex optimization-based method, for estimating the effects of continuous treatments. KOOW finds weights that minimize the worst-case penalized functional covariance between the continuous treatment and the confounders. This material is based upon work supported by the National Science Foundation under Grants Nos. Using data from the Women's Health Initiative observational study, we apply KOOW to evaluate the effect of red meat consumption on blood pressure. Keywords: Independence, continuous actions, policy evaluation, causal inference, optimization, covariate balance 2 1 Introduction The questions that motivate many scientific studies require estimating the effects of continuous treatments. Continuous treatments are usually indexed by doses and their relationships with the outcome are described by dose-response curves.

We introduce a general approach, called test-time training, for improving the performance of predictive models when test and training data come from different distributions. Test-time training turns a single unlabeled test instance into a self-supervised learning problem, on which we update the model parameters before making a prediction on this instance. We show that this simple idea leads to surprising improvements on diverse image classification benchmarks aimed at evaluating robustness to distribution shifts. Theoretical investigations on a convex model reveal helpful intuitions for when we can expect our approach to help.

The automatic classification of applications and services is an invaluable feature for new generation mobile networks. Here, we propose and validate algorithms to perform this task, at runtime, from the raw physical channel of an operative mobile network, without having to decode and/or decrypt the transmitted flows. Towards this, we decode Downlink Control Information (DCI) messages carried within the LTE Physical Downlink Control CHannel (PDCCH). DCI messages are sent by the radio cell in clear text and, in this paper, are utilized to classify the applications and services executed at the connected mobile terminals. Two datasets are collected through a large measurement campaign: one labeled, used to train the classification algorithms, and one unlabeled, collected from four radio cells in the metropolitan area of Barcelona, in Spain. Among other approaches, our Convolutional Neural Network (CNN) classifier provides the highest classification accuracy of 99%. The CNN classifier is then augmented with the capability of rejecting sessions whose patterns do not conform to those learned during the training phase, and is subsequently utilized to attain a fine grained decomposition of the traffic for the four monitored radio cells, in an online and unsupervised fashion.

Dropout methods are a family of stochastic techniques used in neural network training or inference that have generated significant research interest and are widely used in practice. They have been successfully applied in neural network regularization, model compression, and in measuring the uncertainty of neural network outputs. While original formulated for dense neural network layers, recent advances have made dropout methods also applicable to convolutional and recurrent neural network layers. This paper summarizes the history of dropout methods, their various applications, and current areas of research interest. Important proposed methods are described in additional detail.

-- Data generated by cars is growing at an unprecedented scale. As cars gradually become part of the Internet of Things (IoT) ecosystem, several stakeholders discover the value of in-vehicle network logs containing the measurements of the multitude of sensors deployed within the car . This wealth of data is also expected to be exploitable by third parties for the purpose of profiling drivers in order to provide personalized, value-added services. Although several prior works have successfully demonstrated the feasibility of driver re-identification using the in-vehicle network data captured on the vehicle's CAN (Controller Area Network) bus, they inferred the identity of the driver only from known sensor signals (such as the vehicle's speed, brake pedal position, steering wheel angle, etc.) extracted from the CAN messages. However, car manufacturers intentionally do not reveal exact signal location and semantics within CAN logs. We show that the inference of driver identity is possible even with off-the-shelf machine learning techniques without reverse-engineering the CAN protocol. We demonstrate our approach on a dataset of 33 drivers and show that a driver can be re-identified and distinguished from other drivers with an accuracy of 75-85%. I NTRODUCTION Almost all vehicles in use nowadays are equipped with various on-board Electrical Control Units (ECUs), sensors and actuators measuring and controlling the vehicle's speed, acceleration, braking, fuel consumption, battery status, or tire pressure level, among others. Sensors attached to their respective ECUs, whose number ranges from 5 to a hundred per vehicle, continuously generate a vast amount of real-time data. In order to implement complex control tasks for traffic safety or passenger infotainment, these data are then transferred among ECUs and other nodes over the in-vehicle network, most commonly following the Controller Area Network (CAN bus) standard [1]. In addition to being used real-time for automotive control, data are also transferred to car manufacturers through Internet-connected devices or pre-installed modems every half to five minutes.

Obstructive sleep apnea is a serious condition causing a litany of health problems especially in the pediatric population. However, this chronic condition can be treated if diagnosis is possible. The gold standard for diagnosis is an overnight sleep study, which is often unobtainable by many potentially su ff ering from this condition. Hence, we attempt to develop a fast noninvasive diagnostic tool by training a classifier on 2D and 3D facial images of a patient to recognize facial features associated with obstructive sleep apnea. In this comparative study, we consider both persistent homology and geometric shape analysis from the field of computational topology as well as con-volutional neural networks, a powerful method from deep learning whose success in image and specifically facial recognition has already been demonstrated by computer scientists. Keywords: obstructive sleep apnea, machine learning, persistent homology, shape analysis 1. Introduction Obstructive sleep apnea (OSA) is a chronic condition characterized by frequent episodes of upper airway collapse during sleep. Pediatric OSA is a serious health problem as even mild forms of untreated pediatric OSA can cause high blood pressure, changes to the heart, and challenging behaviors, or even alter the childs growth. Unlike adults, the symptoms of childhood-onset OSA are more varied and change with developmental age which creates di fficulties in both the diagnosis and patient management. Prevalence of OSA in children and adolescents is in the range of 1-5%. It is also believed to negatively influence school performance and learning potential. Prompt treatment is a necessity, but long wait times and delays in diagnosis are overly prevalent. The gold standard for diagnosis of pediatric OSA is by overnight polysomnography (PSG) in a hospital or sleep clinic. In many countries, access to PSG is severely limited, and many children do not have confirmation of the diagnosis before treatment. Consequently, some children who do not have OSA will undergo unnecessary surgery to remove their tonsils and adenoids while other children with serious OSA will go untreated. Thus, simple and accessible options to identify children with OSA are greatly needed. A possible simpler approach to diagnosis than PSG might be to examine the structure of a patient's face with the goal of identifying facial features that indicate a high risk for the presence of OSA. Face verification consists of representations of a patient's face that extract important features. A distance measure can be used to determine the similarities and dissimilarities between pairs of faces. Mathematically, the face features lie in a metric space, a space where the distance between two objects can be defined.

Pronounced variability due to the growth of renewable energy sources, flexible loads, and distributed generation is challenging residential distribution systems. This context, motivates well fast, efficient, and robust reactive power control. Real-time optimal reactive power control is possible in theory by solving a non-convex optimization problem based on the exact model of distribution flow. However, lack of high-precision instrumentation and reliable communications, as well as the heavy computational burden of non-convex optimization solvers render computing and implementing the optimal control challenging in practice. Taking a statistical learning viewpoint, the input-output relationship between each grid state and the corresponding optimal reactive power control is parameterized in the present work by a deep neural network, whose unknown weights are learned offline by minimizing the power loss over a number of historical and simulated training pairs. In the inference phase, one just feeds the real-time state vector into the learned neural network to obtain the `optimal' reactive power control with only several matrix-vector multiplications. The merits of this novel statistical learning approach are computational efficiency as well as robustness to random input perturbations. Numerical tests on a 47-bus distribution network using real data corroborate these practical merits.

We show that there is shared structure between different NLP datasets that HUBERT, but not BERT, is able to learn and leverage. Our experiment results show that untangling data-specific semantics from general language structure is key for better transfer among NLP tasks. Built on the Transformer architecture (V aswani et al., 2017), the BERT model (Devlin et al., 2018) has demonstrated great power for providing general-purpose vector embeddings of natural language: its representations have served as the basis of many successful deep Natural Language Processing (NLP) models on a variety of tasks (e.g., Liu et al., 2019a;b; Zhang et al., 2019). Recent studies (Coenen et al., 2019; Hewitt & Manning, 2019; Lin et al., 2019; Tenney et al., 2019) have shown that BERT representations carry considerable information about grammatical structure, which, by design, is a deep and general encapsulation of linguistic information. Symbolic computation over structured symbolic representations such as parse trees has long been used to formalize linguistic knowledge. To strengthen the generality of BERT's representations, we propose to import into its architecture this type of computation. Symbolic linguistic representations support the important distinction between content and form information. The form consists of a structure devoid of content, such as an unlabeled tree, a collection of nodes defined by their structural positions or roles (Newell, 1980), such as root, left-child-of-root, right-child-of-left-child-of root, etc. In a particular linguistic expression such as "Kim referred to herself during the speech", these purely-structural roles are filled with particular content-bearing symbols, including terminal words like Kim and non-terminal categories like NounPhrase . These role fillers have their own identities, which are preserved as they move from role to role across expressions: Kim retains its referent and its semantic properties whether it fills the subject or the object role in a sentence.

In this paper, we present Gamma-LSTM, an enhanced long short term memory (LSTM) unit, to enable learning of hierarchical representations through multiple stages of temporal abstractions. Gamma memory, a hierarchical memory unit, forms the central memory of Gamma-LSTM with gates to regulate the information flow into various levels of hierarchy, thus providing the unit with a control to pick the appropriate level of hierarchy to process the input at a given instant of time. We demonstrate better performance of Gamma-LSTM model regular and stacked LSTMs in two settings (pixel-by-pixel MNIST digit classification and natural language inference) placing emphasis on the ability to generalize over long sequences.

Work in progress generative models written as programs. Conditions on these random variables are imposed through observe statements, while the sample statements define latent variables we seek to draw inference about. Common to the different languages is the existence of an inference backend, which contains one or more general inference methods. Recent research has addressed the task of making repeated inference less computationally expensive, by using upfront computation to reduce the cost of later executions, an approach known as amortized inference (Gershman and Goodman, 2014). One new method called inference compilation (IC) (Le et al., 2017) enables fast inference on arbitrarily complex and non-differentiable generative models. The approximate posterior distribution it learns can be combined with importance sampling at inference time, so that inference is asymptotically correct.