Deep learning training accesses vast amounts of data at high velocity, posing challenges for datasets retrieved over commodity networks and storage devices. We introduce a way to dynamically reduce the overhead of fetching and transporting training data with a method we term Progressive Compressed Records (PCRs). PCRs deviate from previous formats by using progressive compression to convert a single dataset into multiple datasets of increasing fidelity--all without adding to the total dataset size. Empirically, we implement PCRs and evaluate them on a wide range of datasets: ImageNet, HAM10000, Stanford Cars, and CelebA-HQ. Our results show that different tasks can tolerate different levels of compression. PCRs use an on-disk layout that enables applications to efficiently and dynamically access appropriate levels of compression at runtime. In turn, we demonstrate that PCRs can seamlessly enable a 2 speedup in training time on average over baseline formats. Distributed deep learning exploits parallelism to reduce training time, and consists of three key components: the data pipeline (storage), the forward/backward computation (compute), and the variable synchronization (network). However, little attention has been paid toward scaling the storage layer, where training starts and training data is sourced. Unfortunately, hardware trends point to an increasing divide between compute and networking or storage bandwidth (Li et al., 2016; Lim et al., 2019; Kurth et al., 2018). For example, the transportation of data for machine learning is a key factor in the design of modern data centers (Hazelwood et al., 2018), which are expected to be serviced by slow, yet high capacity, storage media for the foreseeable future (David Reinsel, 2018; Cheng et al., 2015; Rosenthal et al., 2012). This, combined with the memory wall--a lack of bandwidth between compute and memory--suggests that, while computation may be sufficient moving forward, the mechanisms for moving data to the compute may not (Wulf & McKee, 1995; Kwon & Rhu, 2018; Hsieh et al., 2017; Zinkevich et al., 2010). The storage pipeline is therefore a natural area to seek improvements in overall training times, which manifest from the storage medium, through the network, and into the compute nodes.

Learning reward functions from data is a promising path towards achieving scalable Reinforcement Learning (RL) for robotics. However, a major challenge in training agents from learned reward models is that the agent can learn to exploit errors in the reward model to achieve high reward behaviors that do not correspond to the intended task. These reward delusions can lead to unintended and even dangerous behaviors. On the other hand, adversarial imitation learning frameworks (Ho & Ermon, 2016) tend to suffer the opposite problem, where the discriminator learns to trivially distinguish agent and expert behavior, resulting in reward models that produce low reward signal regardless of the input state. In this paper, we connect these two classes of reward learning methods to positive-unlabeled (PU) learning, and we show that by applying a large-scale PU learning algorithm to the reward learning problem, we can address both the reward under-and overestimation problems simultaneously. Our approach drastically improves both GAIL and supervised reward learning, without any additional assumptions. While Reinforcement Learning (RL) has shown itself to be a powerful tool for automating control and decision making, hand-specifying reward functions requires significant engineering effort, especially in real-world settings. Recent works have made promising progress in learning reward functions directly from human supervision, such as ratings (Cabi et al., 2019) and behavior preferences (Wilson et al., 2012; Ibarz et al., 2018). However, in practice, these supervisions are expensive to curate and thus often only cover a fraction of the state space. As a result, the learned reward functions may have large errors in the unlabeled states, and policy learning algorithms tend to exploit these errors to achieve extremely high pseudo-reward via unintended behaviors (Amodei et al., 2016). Practical solutions often require a human to provide supervision in the policy training loop iteratively (Christiano et al., 2017; Ibarz et al., 2018), resulting in a even more laborious process. On the other hand, works in Inverse Reinforcement Learning (IRL) propose to infer reward functions directly from expert behaviors (Ng et al., 2000; Ziebart et al., 2008), but scaling these methods to high-dimensional state space remains a challenge. Ho & Ermon (2016), and many followup works show that GAIL can learn complex behaviors even in high-dimensional spaces.

Starck 1 1 Laboratoire AIM, CEA, CNRS, Université Paris-Saclay, Université Paris Diderot, Sorbonne Paris Cité, F-91191 Gif-sur-Yvette, France 2 ONERA - The French Aerospace Lab, 6 chemin de la V auve aux Granges, BP 80100, FR-91123 P ALAISEAU cedex, France November 4, 2019 ABSTRACT Deconvolution of large survey images with millions of galaxies requires to develop a new generation of methods which can take into account a space variant Point Spread Function (PSF) and have to be at the same time accurate and fast. We investigate in this paper how Deep Learning (DL) could be used to perform this task. We employ a U-NET Deep Neural Network (DNN) architecture to learn in a supervised setting parameters adapted for galaxy image processing and study two strategies for deconvolution. The first approach is a post-processing of a mere Tikhonov deconvolution with closed form solution and the second one is an iterative deconvolution framework based on the Alternating Direction Method of Multipliers (ADMM). Our numerical results based on GREA T3 simulations with realistic galaxy images and PSFs show that our two approaches outperforms standard techniques based on convex optimization, whether assessed in galaxy image reconstruction or shape recovery. The approach based on Tikhonov deconvolution leads to the most accurate results except for ellipticity errors at high signal to noise ratio where the ADMM approach performs slightly better, is also more computation-time e fficient to process a large number of galaxies, and is therefore recommended in this scenario. Methods:statistical, Methods:data analysis, Methods:numerical 1. Introduction Deconvolution of large galaxy survey images requires to take into account spatial-variation of the Point Spread Function (PSF) across the field of view. The PSF field is usually estimated beforehand, via parametric models and simulations as in Krist et al. (2011) or directly estimated from the (noisy) observations of stars in the field of view (Bertin 2011; Kuijken et al. 2015; Zuntz et al. 2018; Mboula et al. 2016; Schmitz et al. 2019). Even with the "perfect" knowledge of the PSF, this ill-posed deconvolution problem is challenging, in particular due to the size of the image to process. Starck et al. (2000) proposed an Object-Oriented Deconvolution, consisting in first detecting galaxies and then deconvolving each object independently. Following this idea, Farrens et al. (2017) introduced a space-variant deconvolution approach for galaxy images, based on two regularization strategies: using either a sparse prior in a transformed domain (Starck et al. 2015a) or trying to learn unsupervisedly a low-dimensional subspace for galaxy representation using a low-rank prior on the recovered galaxy images.

With the rapid adoption of machine learning systems in sensitive applications, there is an increasing need to make black-box models explainable. Often we want to identify an influential group of training samples in a particular test prediction. Existing influence functions tackle this problem by using first-order approximations of the effect of removing a sample from the training set on model parameters. To compute the influence of a group of training samples (rather than an individual point) in model predictions, the change in optimal model parameters after removing that group from the training set can be large. Thus, in such cases, the first-order approximation can be loose. In this paper, we address this issue and propose second-order influence functions for identifying influential groups in test-time predictions. For linear models and across different sizes of groups, we show that using the proposed second-order influence function improves the correlation between the computed influence values and the ground truth ones. For nonlinear models based on neural networks, we empirically show that none of the existing first-order and the proposed second-order influence functions provide proper estimates of the ground-truth influences over all training samples. We empirically study this phenomenon by decomposing the influence values over contributions from different eigenvectors of the Hessian of the trained model.

We present a novel framework to deal with relation extraction tasks in cases where there is complete lack of supervision, either in the form of gold annotations, or relations from a knowledge base. Our approach leverages syntactic parsing and pre-trained word embeddings to extract few but precise relations,which are then used to annotate a larger cor-pus, in a manner identical to distant supervision. The resulting data set is employed to fine tune a pre-trained BERT model in order to perform relation extraction. Empirical evaluation on four data sets from the biomedical domain shows that our method significantly outperforms two simple baselines for unsupervised relation extraction and, even if not using any supervision at all, achieves slightly worse results than the state-of-the-art in three out of four data sets. Importantly, we show that it is possible to successfully fine tune a large pre-trained language model with noisy data, as op-posed to previous works that rely on gold data for fine tuning.

In this paper we are concerned with the learnability of energies from data obtained by observing time evolutions of their critical points starting at random initial equilibria. As a byproduct of our theoretical framework we introduce the novel concept of mean-field limit of critical point evolutions and of their energy balance as a new form of transport. We formulate the energy learning as a variational problem, minimizing the discrepancy of energy competitors from fulfilling the equilibrium condition along any trajectory of critical points originated at random initial equilibria. By Gamma-convergence arguments we prove the convergence of minimal solutions obtained from finite number of observations to the exact energy in a suitable sense. The abstract framework is actually fully constructive and numerically implementable. Hence, the approximation of the energy from a finite number of observations of past evolutions allows to simulate further evolutions, which are fully data-driven. As we aim at a precise quantitative analysis, and to provide concrete examples of tractable solutions, we present analytic and numerical results on the reconstruction of an elastic energy for a one-dimensional model of thin nonlinear-elastic rod.

We introduce a fully stochastic gradient based approach to Bayesian optimal experimental design (BOED). This is achieved through the use of variational lower bounds on the expected information gain (EIG) of an experiment that can be simultaneously optimized with respect to both the variational and design parameters. This allows the design process to be carried out through a single unified stochastic gradient ascent procedure, in contrast to existing approaches that typically construct an EIG estimator on a pointwise basis, before passing this estimator to a separate optimizer. We show that this, in turn, leads to more efficient BOED schemes and provide a number of a different variational objectives suited to different settings. Furthermore, we show that our gradient-based approaches are able to provide effective design optimization in substantially higher dimensional settings than existing approaches.

Learning the causal-interaction network of multivariate Hawkes processes is a useful task in many applications. Maximum-likelihood estimation is the most common approach to solve the problem in the presence of long observation sequences. However, when only short sequences are available, the lack of data amplifies the risk of overfitting and regularization becomes critical. Due to the challenges of hyper-parameter tuning, state-of-the-art methods only parameterize regularizers by a single shared hyper-parameter, hence limiting the power of representation of the model. To solve both issues, we develop in this work an efficient algorithm based on variational expectation-maximization. Our approach is able to optimize over an extended set of hyper-parameters. It is also able to take into account the uncertainty in the model parameters by learning a posterior distribution over them. Experimental results on both synthetic and real datasets show that our approach significantly outperforms state-of-the-art methods under short observation sequences.

The adaptive optimizer for training neural networks has continually evolved to overcome the limitations of the previously proposed adaptive methods. Recent studies have found the rare counterexamples that Adam cannot converge to the optimal point. Those counterexamples reveal the distortion of Adam due to a small second momentum from a small gradient. Unlike previous studies, we show Adam cannot keep closer to the optimal point for not only the counterexamples but also a general convex region when the effective learning rate exceeds the certain bound. Subsequently, we propose an algorithm that overcomes Adam's limitation and ensures that it can reach and stay at the optimal point region.

Nonlinear independent component analysis (ICA) is a general framework for unsupervised representation learning, and aimed at recovering the latent variables in data. Recent practical methods perform nonlinear ICA by solving a series of classification problems based on logistic regression. However, it is well-known that logistic regression is vulnerable to outliers, and thus the performance can be strongly weakened by outliers. In this paper, we first theoretically analyze nonlinear ICA models in the presence of outliers. Our analysis implies that estimation in nonlinear ICA can be seriously hampered when outliers exist on the tails of the (noncontaminated) target density, which happens in a typical case of contamination by outliers. We develop two robust nonlinear ICA methods based on the {\gamma}-divergence, which is a robust alternative to the KL-divergence in logistic regression. The proposed methods are shown to have desired robustness properties in the context of nonlinear ICA. We also experimentally demonstrate that the proposed methods are very robust and outperform existing methods in the presence of outliers. Finally, the proposed method is applied to ICA-based causal discovery and shown to find a plausible causal relationship on fMRI data.