Many applications of machine learning and optimization operate on data streams. While these datasets are fundamental to fuel decision-making algorithms, often they contain sensitive information about individuals, and their usage poses significant privacy risks. Motivated by an application in energy systems, this paper presents OptStream, a novel algorithm for releasing differentially private data streams under the w-event model of privacy. OptStream is a 4-step procedure consisting of sampling, perturbation, reconstruction, and post-processing modules. First, the sampling module selects a small set of points to access in each period of interest. Then, the perturbation module adds noise to the sampled data points to guarantee privacy. Next, the reconstruction module re-assembles non-sampled data points from the perturbed sample points. Finally, the post-processing module uses convex optimization over the privacy-preserving output of the previous modules, as well as the privacy-preserving answers of additional queries on the data stream, to improve accuracy by redistributing the added noise. OptStream is evaluated on a test case involving the release of a real data stream from the largest European transmission operator. Experimental results show that OptStream may not only improve the accuracy of state-of-the-art methods by at least one order of magnitude but also supports accurate load forecasting on the privacy-preserving data.

We propose a novel neural topic model in the Wasserstein autoencoders (WAE) framework. Unlike existing variational autoencoder based models, we directly enforce Dirichlet prior on the latent document-topic vectors. We exploit the structure of the latent space and apply a suitable kernel in minimizing the Maximum Mean Discrepancy (MMD) to perform distribution matching. We discover that MMD performs much better than the Generative Adversarial Network (GAN) in matching high dimensional Dirichlet distribution. We further discover that incorporating randomness in the encoder output during training leads to significantly more coherent topics. To measure the diversity of the produced topics, we propose a simple topic uniqueness metric. Together with the widely used coherence measure NPMI, we offer a more wholistic evaluation of topic quality. Experiments on several real datasets show that our model produces significantly better topics than existing topic models.

Minimizing a convex function of a measure with a sparsity-inducing penalty is a typical problem arising, e.g., in sparse spikes deconvolution or two-layer neural networks training. We show that this problem can be solved by discretizing the measure and running non-convex gradient descent on the positions and weights of the particles. For measures on a $d$-dimensional manifold and under some non-degeneracy assumptions, this leads to a global optimization algorithm with a complexity scaling as $\log(1/\epsilon)$ in the desired accuracy $\epsilon$, instead of $\epsilon^{-d}$ for convex methods. The key theoretical tools are a local convergence analysis in Wasserstein space and an analysis of a perturbed mirror descent in the space of measures. Our bounds involve quantities that are exponential in $d$ which is unavoidable under our assumptions.

In modern building infrastructures, the chance to devise adaptive and unsupervised data-driven health monitoring systems is gaining in popularity due to the large availability of data from low-cost sensors with internetworking capabilities. In particular, deep learning provides the tools for processing and analyzing this unprecedented amount of data efficiently. The main purpose of this paper is to combine the recent advances of Deep Learning (DL) and statistical analysis on structural health monitoring (SHM) to develop an accurate classification tool able to discriminate among different acoustic emission events (cracks) by means of the identification of tensile, shear and mixed modes. The applications of DL in SHM systems is described by using the concept of Bidirectional Long Short Term Memory. We investigated on effective event descriptors to capture the unique characteristics from the different types of modes. Among them, Spectral Kurtosis and Spectral L2/L1 Norm exhibit distinctive behavior and effectively contributed to the learning process. This classification will contribute to unambiguously detect incipient damages, which is advantageous to realize predictive maintenance. Tests on experimental results confirm that this method achieves accurate classification (92%) capabilities of crack events and can impact on the design of future SHM technologies.

Methane is considered being a good choice as a propellant for future reusable launch systems. However, the heat transfer prediction for supercritical methane flowing in cooling channels of a regeneratively cooled combustion chamber is challenging. Because accurate heat transfer predictions are essential to design reliable and efficient cooling systems, heat transfer modeling is a fundamental issue to address. Advanced computational fluid dynamics (CFD) calculations achieve sufficient accuracy, but the associated computational cost prevents an efficient integration in optimization loops. Surrogate models based on artificial neural networks (ANNs) offer a great speed advantage. It is shown that an ANN, trained on data extracted from samples of CFD simulations, is able to predict the maximum wall temperature along straight rocket engine cooling channels using methane with convincing precision. The combination of the ANN model with simple relations for pressure drop and enthalpy rise results in a complete reduced order model, which can be used for numerically efficient design space exploration and optimization.

We develop a stochastic whole-brain and body simulator of the nematode roundworm Caenorhabditis elegans (C. elegans) and show that it is sufficiently regularizing to allow imputation of latent membrane potentials from partial calcium fluorescence imaging observations. This is the first attempt we know of to "complete the circle," where an anatomically grounded whole-connectome simulator is used to impute a time-varying "brain" state at single-cell fidelity from covariates that are measurable in practice. The sequential Monte Carlo (SMC) method we employ not only enables imputation of said latent states but also presents a strategy for learning simulator parameters via variational optimization of the noisy model evidence approximation provided by SMC. Our imputation and parameter estimation experiments were conducted on distributed systems using novel implementations of the aforementioned techniques applied to synthetic data of dimension and type representative of that which are measured in laboratories currently.

The legacy measurements of the LHC will require analyzing high-dimensional event data for subtle kinematic signatures, which is challenging for established analysis methods. Recently, a powerful family of multivariate inference techniques that leverage both matrix element information and machine learning has been developed. This approach neither requires the reduction of high-dimensional data to summary statistics nor any simplifications to the underlying physics or detector response. In this paper we introduce MadMiner, a Python module that streamlines the steps involved in this procedure. Wrapping around MadGraph5_aMC and Pythia 8, it supports almost any physics process and model. To aid phenomenological studies, the tool also wraps around Delphes 3, though it is extendable to a full Geant4-based detector simulation. We demonstrate the use of MadMiner in an example analysis of dimension-six operators in ttH production, finding that the new techniques substantially increase the sensitivity to new physics.

In an era when the performance of a single compute device plateaus, software must be designed to scale on a massively parallel system for better runtime performance. However, the commonly used back-propagation (BP) algorithm imposes a strong sequential dependency in the process of gradient computation. Under model parallelism, BP has a theoretical step complexity of $\Theta (n)$ which hinders its scalability in a parallel computing environment, where $n$ represents the number of compute devices into which a model is partitioned. In this work, we restructure such dependency and reformulate BP into a scan operation which is scaled by our modified version of the Blelloch scan algorithm. Our algorithm is able to achieve a theoretical step complexity of $\Theta (\log n)$. We perform an in-depth performance analysis and identify the challenges of deploying our algorithm in a practical setting, along with a variety of approaches to tackle such challenges. We demonstrate the scalability benefits of our algorithm in the use case of retraining pruned networks.

We implement machine learning algorithms to nuclear data. These algorithms are purely data driven and generate models that are capable to capture intricate trends. Gradient boosted trees algorithm is employed to generate a trained model from existing nuclear data, which is used for prediction for data of damping parameter, shell correction energies, quadrupole deformation, pairing gaps, level densities and giant dipole resonance for large number of nuclei. We, in particular, predict level density parameter for superheavy elements which is of great current interest. The predictions made by the machine learning algorithm is found to have standard deviation from 0.00035 to 0.73.

This article was originally published on the National Center for Supercomputing Applications website. In 2007, the Sloan Digital Sky Survey (SDSS) launched a citizen science campaign called Galaxy Zoo to enlist the public's help in classifying the hundreds of thousands of galaxy images captured by an optical telescope. Through this highly successful crowdsourcing effort, volunteers reviewed the images online to help determine whether each galaxy had a spiral or elliptical structure. Leveraging data generated by the Galaxy Zoo project, a team of scientists is now applying the power of artificial intelligence (AI) and high-performance supercomputers to accelerate efforts to analyze the increasingly massive datasets produced by ongoing and future cosmological surveys. In a new study, researchers from the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign and the Argonne Leadership Computing Facility (ALCF) at the U.S. Department of Energy's (DOE) Argonne National Laboratory have developed a novel combination of deep learning methods to provide a highly accurate approach to classifying hundreds of millions of unlabeled galaxies.