Australia is home to more than 10,000 beaches, ranging from a few dozen metres to hundreds of kilometres long. But increasingly, these beaches are slowly disappearing before our eyes. "Beaches across Australia are eroding, simply because waves come in pull sand away -- and big storm surges pull more sand away," IBM Systems Data Scientist Dr Adam Makarucha told the Gartner Application Architecture, Development, and Integration Summit in Sydney. While the likes of Gold Coast Council have invested AU$14 million into rehabilitation projects -- such as one for a 12km stretch of beach, equating to more than a million dollars per kilometre -- Makarucha said prevention is more viable than rehabilitation. Makarucha said the best way to prevent beach erosion is to look to a natural defence, such as seagrass.

The convergence of AI and HPC has created a fertile venue that is ripe for imaginative researchers -- versed in AI technology -- to make a big impact in a variety of scientific fields. From new hardware to new computational approaches, the true impact of deep- and machine learning on HPC is, in a word, "everywhere". Just as technology changes in the personal computer market brought about a revolution in the design and implementation of the systems and algorithms used in high performance computing (HPC), so are recent technology changes in machine learning bringing about an AI revolution in the HPC community. Expect new HPC analytic techniques including the use of GANs (Generative Adversarial Networks) in physics-based modeling and simulation, as well as reduced precision math libraries such as NLAFET and HiCMA to revolutionise many fields of research. Other benefits of the convergence of AI and HPC include the physical instantiation of data flow architectures in FPGAs and ASICs, plus the development of powerful data analytic services.

Earthquake Alert! 6.7 temblor, epicenter 3.8 miles west of Ventura, California--impact will be in eleven minutes--evacuate, evacuate!" While you run to the hall closet to grab your earthquake kit, you shout out: "Alexa, where is my emergency evac location?" Walk north to Wilshire, then take a left on Warner," she responds. As you and your neighbors pour into the building stairwell, you hear audio from a phone: "Google Earth Q estimates substantial potential for structural damage in the West San Fernando Valley and Coastal West Los Angeles to pre-2006 code dwellings and buildings. Most of West LA will experience total loss of power for anywhere from six to twenty-four hours in duration."

With technological advancement, IT technology developers are making efforts to develop software that can ease the physical work life. One such advancement in technology is intelligent automation. The intelligent automation is a combination of automation and artificial intelligence. This new technology has revolutionized the way data is handled and processed. The intelligent automation system determines and synthesizes a massive amount of information, automates the business and operational workflows and adapts it.

While generative models have shown great success in generating high-dimensional samples conditional on low-dimensional descriptors (learning e.g. stroke thickness in MNIST, hair color in CelebA, or speaker identity in Wavenet), their generation out-of-sample poses fundamental problems. The conditional variational autoencoder (CVAE) as a simple conditional generative model does not explicitly relate conditions during training and, hence, has no incentive of learning a compact joint distribution across conditions. We overcome this limitation by matching their distributions using maximum mean discrepancy (MMD) in the decoder layer that follows the bottleneck. This introduces a strong regularization both for reconstructing samples within the same condition and for transforming samples across conditions, resulting in much improved generalization. We refer to the architecture as \emph{transformer} VAE (trVAE). Benchmarking trVAE on high-dimensional image and tabular data, we demonstrate higher robustness and higher accuracy than existing approaches. In particular, we show qualitatively improved predictions for cellular perturbation response to treatment and disease based on high-dimensional single-cell gene expression data, by tackling previously problematic minority classes and multiple conditions. For generic tasks, we improve Pearson correlations of high-dimensional estimated means and variances with their ground truths from 0.89 to 0.97 and 0.75 to 0.87, respectively.

Recent advances in pre-training huge models on large amounts of text through self supervision have obtained state-of-the-art results in various natural language processing tasks. However, these huge and expensive models are difficult to use in practise for downstream tasks. Some recent efforts use knowledge distillation to compress these models. However, we see a gap between the performance of the smaller student models as compared to that of the large teacher. In this work, we leverage large amounts of in-domain unlabeled transfer data in addition to a limited amount of labeled training instances to bridge this gap. We show that simple RNN based student models even with hard distillation can perform at par with the huge teachers given the transfer set. The student performance can be further improved with soft distillation and leveraging teacher intermediate representations. We show that our student models can compress the huge teacher by up to 26x while still matching or even marginally exceeding the teacher performance in low-resource settings with small amount of labeled data.

Recently, there has been significant interest in linear regression in the situation where predictors and responses are not observed in matching pairs corresponding to the same statistical unit as a consequence of separate data collection and uncertainty in data integration. Mismatched pairs can considerably impact the model fit and disrupt the estimation of regression parameters. In this paper, we present a method to adjust for such mismatches under ``partial shuffling" in which a sufficiently large fraction of (predictors, response)-pairs are observed in their correct correspondence. The proposed approach is based on a pseudo-likelihood in which each term takes the form of a two-component mixture density. Expectation-Maximization schemes are proposed for optimization, which (i) scale favorably in the number of samples, and (ii) achieve excellent statistical performance relative to an oracle that has access to the correct pairings as certified by simulations and case studies. In particular, the proposed approach can tolerate considerably larger fraction of mismatches than existing approaches, and enables estimation of the noise level as well as the fraction of mismatches. Inference for the resulting estimator (standard errors, confidence intervals) can be based on established theory for composite likelihood estimation. Along the way, we also propose a statistical test for the presence of mismatches and establish its consistency under suitable conditions.

Convolutional neural networks (CNNs) have achieved breakthrough performances in a wide range of applications including image classification, semantic segmentation, and object detection. Previous research on characterizing the generalization ability of neural networks mostly focuses on fully connected neural networks (FNNs), regarding CNNs as a special case of FNNs without taking into account the special structure of convolutional layers. In this work, we propose a tighter generalization bound for CNNs by exploiting the sparse and permutation structure of its weight matrices. As the generalization bound relies on the spectral norm of weight matrices, we further study spectral norms of three commonly used convolution operations including standard convolution, depthwise convolution, and pointwise convolution. Theoretical and experimental results both demonstrate that our bounds for CNNs are tighter than existing bounds.

Silas: High Performance, Explainable and V erifiable Machine Learning Hadrien Bride, Zh e H ou Griffith University, Nathan, Brisbane, Australia Jie Dong Dependable Intelligence Pty Ltd, Brisbane, Australia Jin Song Dong National University of Singapore, Singapore Ali Mirjalili Griffith University, Nathan, Brisbane, AustraliaAbstract This paper introduces a new classification tool named Silas, which is built to provide a more transparent and dependable data analytics service. A focus of Silas is on providing a formal foundation of decision trees in order to support logical analysis and verification of learned prediction models. This paper describes the distinct features of Silas: The Model Audit module formally verifies the prediction model against user specifications, the Enforcement Learning module trains prediction models that are guaranteed correct, the Model Insight and Prediction Insight modules reason about the prediction model and explain the decision-making of predictions. We also discuss implementation details ranging from programming paradigm to memory management that help achieve high-performance computation.1. Introduction Machine learning has enjoyed great success in many research areas and industries, including entertainment [1], self-driving cars [2], banking [3], medical diagnosis [4], shopping [5], and among many others. However, the wide adoption of machine learn-Preprint submitted to Elsevier October 4, 2019 arXiv:1910.01382v1 The ramifications of the black-box approach are multifold. First, it may lead to unexpected results that are only observable after the deployment of the algorithm. For instance, Amazon's Alexa offered porn to a child [6], a self-driving car had a deadly accident [7], etc. Some of these accidents result in lawsuits or even lost lives, the cost of which is immeasurable. Second, it prevents the adoption in some applications and industries where an explanation is mandatory or certain specifications must be satisfied. For example, in some countries, it is required by law to give a reason why a loan application is rejected. In recent years, eXplainable AI (XAI) has been gaining attention, and there is a surge of interest in studying how prediction models work and how to provide formal guarantees for the models. A common theme in this space is to use statistical methods to analyse prediction models.

Deep neural network image classifiers are reported to be susceptible to adversarial evasion attacks, which use carefully crafted images created to mislead a classifier. Recently, various kinds of adversarial attack methods have been proposed, most of which focus on adding small perturbations to input images. Despite the success of existing approaches, the way to generate realistic adversarial images with small perturbations remains a challenging problem. In this paper, we aim to address this problem by proposing a novel adversarial method, which generates adversarial examples by imposing not only perturbations but also spatial distortions on input images, including scaling, rotation, shear, and translation. As humans are less susceptible to small spatial distortions, the proposed approach can produce visually more realistic attacks with smaller perturbations, able to deceive classifiers without affecting human predictions. We learn our method by amortized techniques with neural networks and generate adversarial examples efficiently by a forward pass of the networks. Extensive experiments on attacking different types of non-robustified classifiers and robust classifiers with defence show that our method has state-of-the-art performance in comparison with advanced attack parallels.