This paper develops a new class of nonconvex regularizers for low-rank matrix recovery. Many regularizers are motivated as convex relaxations of the matrix rank function. Our new factor group-sparse regularizers are motivated as a relaxation of the number of nonzero columns in a factorization of the matrix. These nonconvex regularizers are sharper than the nuclear norm; indeed, we show they are related to Schatten-$p$ norms with arbitrarily small $0 < p \leq 1$. Moreover, these factor group-sparse regularizers can be written in a factored form that enables efficient and effective nonconvex optimization; notably, the method does not use singular value decomposition. We provide generalization error bounds for low-rank matrix completion which show improved upper bounds for Schatten-$p$ norm reglarization as $p$ decreases. Compared to the max norm and the factored formulation of the nuclear norm, factor group-sparse regularizers are more efficient, accurate, and robust to the initial guess of rank. Experiments show promising performance of factor group-sparse regularization for low-rank matrix completion and robust principal component analysis.

The paper proposes an estimator to make inference on key features of heterogeneous treatment effects sorted by impact groups (GATES) for non-randomised experiments. Observational studies are standard in policy evaluation from labour markets, educational surveys, and other empirical studies. To control for a potential selection-bias we implement a doubly-robust estimator in the first stage. Keeping the flexibility to use any machine learning method to learn the conditional mean functions as well as the propensity score we also use machine learning methods to learn a function for the conditional average treatment effect. The group average treatment effect is then estimated via a parametric linear model to provide p-values and confidence intervals. The result is a best linear predictor for effect heterogeneity based on impact groups. Cross-splitting and averaging for each observation is a further extension to avoid biases introduced through sample splitting. The advantage of the proposed method is a robust estimation of heterogeneous group treatment effects under mild assumptions, which is comparable with other models and thus keeps its flexibility in the choice of machine learning methods. At the same time, its ability to deliver interpretable results is ensured.

Designing effective architectures is one of the key factors behind the success of deep neural networks. Existing deep architectures are either manually designed or automatically searched by some Neural Architecture Search (NAS) methods. However, even a well-searched architecture may still contain many non-significant or redundant modules or operations (e.g., convolution or pooling), which may not only incur substantial memory consumption and computation cost but also deteriorate the performance. Thus, it is necessary to optimize the operations inside an architecture to improve the performance without introducing extra computation cost. Unfortunately, such a constrained optimization problem is NP-hard. To make the problem feasible, we cast the optimization problem into a Markov decision process (MDP) and seek to learn a Neural Architecture Transformer (NAT) to replace the redundant operations with the more computationally efficient ones (e.g., skip connection or directly removing the connection). Based on MDP, we learn NAT by exploiting reinforcement learning to obtain the optimization policies w.r.t. different architectures. To verify the effectiveness of the proposed strategies, we apply NAT on both hand-crafted architectures and NAS based architectures. Extensive experiments on two benchmark datasets, i.e., CIFAR-10 and ImageNet, demonstrate that the transformed architecture by NAT significantly outperforms both its original form and those architectures optimized by existing methods.

Efficient exploration for automatic subgoal discovery is a challenging problem in Hierarchical Reinforcement Learning (HRL). In this paper, we show that intrinsic motivation learning increases the efficiency of exploration, leading to successful subgoal discovery. We introduce a model-free subgoal discovery method based on unsupervised learning over a limited memory of agent's experiences during intrinsic motivation. Additionally, we offer a unified approach to learning representations in model-free HRL.

Machine learning technologies are increasingly developed for use in healthcare. While research communities have focused on creating state-of-the-art models, there has been less focus on real world implementation and the associated challenges to accuracy, fairness, accountability, and transparency that come from actual, situated use. Serious questions remain under examined regarding how to ethically build models, interpret and explain model output, recognize and account for biases, and minimize disruptions to professional expertise and work cultures. We address this gap in the literature and provide a detailed case study covering the development, implementation, and evaluation of Sepsis Watch, a machine learning-driven tool that assists hospital clinicians in the early diagnosis and treatment of sepsis. We, the team that developed and evaluated the tool, discuss our conceptualization of the tool not as a model deployed in the world but instead as a socio-technical system requiring integration into existing social and professional contexts. Rather than focusing on model interpretability to ensure a fair and accountable machine learning, we point toward four key values and practices that should be considered when developing machine learning to support clinical decision-making: rigorously define the problem in context, build relationships with stakeholders, respect professional discretion, and create ongoing feedback loops with stakeholders. Our work has significant implications for future research regarding mechanisms of institutional accountability and considerations for designing machine learning systems. Our work underscores the limits of model interpretability as a solution to ensure transparency, accuracy, and accountability in practice. Instead, our work demonstrates other means and goals to achieve FATML values in design and in practice.

Representation learning is critical to the success of modern large-scale reinforcement learning systems. Previous works show that sparse representation can effectively reduce catastrophic interference and hence provide relatively stable and consistent boostrap targets when training reinforcement learning algorithms. Tile coding is a well-known sparse feature generation method in reinforcement learning. However, its application is largely restricted to small, low dimensional domains, as its computational and memory requirement grows exponentially as dimension increases. This paper proposes a simple and novel tile coding operation---deep tile coder, which adapts tile coding into deep learning setting, and can be easily scaled to high dimensional problems. The key distinction of our method with previous sparse representation learning method is that, we generate sparse feature by construction, while most previous works focus on designing regularization techniques. We are able to theoretically guarantee sparsity and importantly, our method ensures sparsity from the beginning of learning, without the need of tuning regularization weight. Furthermore, our approach maps from low dimension feature space to high dimension sparse feature space without introducing any additional training parameters. Our empirical demonstration covers classic discrete action control and Mujoco continuous robotics control problems. We show that reinforcement learning algorithms equipped with our deep tile coder achieves superior performance. To our best knowledge, our work is the first to demonstrate successful application of sparse representation learning method in online deep reinforcement learning algorithms for challenging tasks without using a target network.

Adversarial Inverse Reinforcement Learning for Decision Making in Autonomous Driving Pin Wang 1, Dapeng Liu 2, 3, Jiayu Chen 4, and Ching-Y ao Chan 1 Abstract -- Generative Adversarial Imitation Learning (GAIL) is an efficient way to learn sequential control strategies from demonstration. Adversarial Inverse Reinforcement Learning (AIRL) is similar to GAIL but also learns a reward function at the same time and has better training stability. In previous work, however, AIRL has mostly been demonstrated on robotic control in artificial environments. In this paper, we apply AIRL to a practical and challenging problem - the decision-making in autonomous driving, and also augment AIRL with a semantic reward to improve its performance. We use four metrics to evaluate its learning performance in a simulated driving environment. Results show that the vehicle agent can learn decent decision-making behaviors from scratch, and can reach a level of performance comparable with that of an expert. Additionally, the comparison with GAIL shows that AIRL converges faster, achieves better and more stable performance than GAIL. I. INTRODUCTION The application of Reinforcement Learning (RL) in robotics has been very fruitful in recent years.

{\alpha}{\mu} is an anytime heuristic search algorithm for incomplete information games that assumes perfect information for the opponents. {\alpha}{\mu} addresses the strategy fusion and non-locality problems encountered by Perfect Information Monte Carlo sampling. In this paper {\alpha}{\mu} is applied to the game of Bridge.

Knowledge Graph (KG) embedding has attracted more attention in recent years. Most of KG embedding models learn from time-unaware triples. However, the inclusion of temporal information beside triples would further improve the performance of a KGE model. In this regard, we propose A TiSE, a temporal KG embedding model which incorporates time information into entity/relation representations by using A dditive Time Se ries decomposition. Moreover, considering the temporal uncertainty during the evolution of entity/relation representations over time, we map the representations of temporal KGs into the space of multidimensional Gaussian distributions. The mean of each entity/relation embedding at a time step shows the current expected position, whereas its covariance (which is temporally stationary) represents its temporal uncertainty. Experimental results show that A TiSE not only achieves the state-of-the-art on link prediction over temporal KGs, but also can predict the occurrence time of facts with missing time annotations, as well as the existence of future events. To the best of our knowledge, no other model is capable to perform all these tasks.

The significant interest in combining logic and probability for reasoning in uncertain, relational domains has led to a multitude of formalisms, inc luding the family of probabilistic logic programming (PLP) languages based on the dis tribution semantics [Sato, 1995] with languages and systems such as PRISM [Sato, 1995], ICL [Poole, 2008], ProbLog [De Raedt et al., 2007; Fierens et al., 2015] and PIT A [Riguzzi and Swift, 2011]. State-of-the-art inference for PLP uses a reduction to weig hted model counting (WMC) [Chavira and Darwiche, 2008], where the dependency structure of the logic program a nd the queries is first transformed into a propositional formula in a suitable form at that supports efficient WMC. While the details of this transformation differ across approaches, a key part of it is determining the relevant ground program with respect t o the queries of interest, i.e., all groundings of rules that contribute to some deriva tion of a query. This grounding step has received little attention, as its cost is domina ted by the cost of constructing the propositional formula in typical PLP benchmarks that op erate on biological, social or hyperlink networks, where formulas are complex. However, it has been observed 1 that the grounding step is the bottleneck that often makes it impossible to apply PLP inference in the context of ontology-based data access over probabilistic data (pOBDA) [Schoenfisch and Stuckenschmidt, 2017; van Bremen et al., 20 19], where determining the relevant grounding explores a large search space, but on ly small parts of this space contribute to the formulas.