A study by McKinsey & Company found that AI-driven quality testing can increase productivity by up to 50% and defect detection rates by up to 90% compared to human inspection. Though machines with automated optical inspection (AOI), powered by machine vision, have replaced most of the manual processes in the modern assembly line, quality control still remains a huge and costly challenge. The European Commission claims that in some industries 50% of production can be abandoned due to defects, and the defect rate can reach up to 90% in complex production environments. The critical limitation with machine learning AOI systems is in disclosing surface defects where even a slight variant (often invisible to the human eye) can hamper the entire production run and render hundreds to thousands of products useless before the defect is discovered. The economic impact can be devastating.

Deep learning models like Google's BERT and the new OpenAI GPT-3 have brought machines much closer to approximating human understanding. The keyword here is "approximating" because these deep learning models don't actually understand the text they see. While not perfect, they have become much better at predicting what words might come next in a given sentence or search string. Does this mean we're getting close to true artificial intelligence (AI)? Not yet, although machines will soon be able to do the heavy lifting when it comes to data analysis so that all we will have to do is step in and interpret the results.

The artificial intelligence (AI) revolution has been underway since about 2016. As a result of great increases in computational power, AI no longer belongs to the realm of media hype and science fiction. Today, AI offers concrete benefits in all areas of engineering, manufacturing, and operations. From deep neural networks and long short-term memory (LSTM) algorithms to reinforcement and physics informed neural networks (PINN), the possibilities are endless, and the real-world applications are only just beginning to truly be exploited. Industry data holds gold nuggets; AI is they key to finding and using them.

In the present work, we explore the capability of artificial neural networks (ANN) to predict the closure terms for large eddy simulations (LES) solely from coarse-scale data. To this end, we derive a consistent framework for LES closure models, with special emphasis laid upon the incorporation of implicit discretization-based filters and numerical approximation errors. We investigate implicit filter types, which are inspired by the solution representation of discontinuous Galerkin and finite volume schemes and mimic the behaviour of the discretization operator, and a global Fourier cutoff filter as a representative of a typical explicit LES filter. Within the perfect LES framework, we compute the exact closure terms for the different LES filter functions from direct numerical simulation results of decaying homogeneous isotropic turbulence. Multiple ANN with a multilayer perceptron (MLP) or a gated recurrent unit (GRU) architecture are trained to predict the computed closure terms solely from coarse-scale input data. For the given application, the GRU architecture clearly outperforms the MLP networks in terms of accuracy, whilst reaching up to 99.9% cross-correlation between the networks' predictions and the exact closure terms for all considered filter functions. The GRU networks are also shown to generalize well across different LES filters and resolutions. The present study can thus be seen as a starting point for the investigation of data-based modeling approaches for LES, which not only include the physical closure terms, but account for the discretization effects in implicitly filtered LES as well.

Graph neural networks (GNNs) have become a popular approach to integrating structural inductive biases into NLP models. However, there has been little work on interpreting them, and specifically on understanding which parts of the graphs (e.g. In this work, we introduce a post-hoc method for interpreting the predictions of GNNs which identifies unnecessary edges. Given a trained GNN model, we learn a simple classifier that, for every edge in every layer, predicts if that edge can be dropped. We use our technique as an attribution method to analyze GNN models for two tasks - question answering and semantic role labeling - providing insights into the information flow in these models. We show that we can drop a large proportion of edges without deteriorating the performance of the model, while we can analyse the remaining edges for interpreting model predictions. Graph Neural Networks (GNNs) have in recent years been shown to provide a scalable and highly performant means of incorporating linguistic information and other structural biases into NLP models. While GNNs often yield strong performance, such models are complex, and it can be difficult to understand the'reasoning' behind their predictions. For NLP practitioners, it is highly desirable to know which linguistic information a given model encodes and how that encoding happens (Jumelet & Hupkes, 2018; Giulianelli et al., 2018; Goldberg, 2019). The difficulty in interpreting GNNs represents a barrier to such analysis.

An estimated 180 papers focusing on deep learning and EHR were published between 2010 and 2018. Despite the common workflow structure appearing in these publications, no trusted and verified software framework exists, forcing researchers to arduously repeat previous work. In this paper, we propose Cardea, an extensible open-source automated machine learning framework encapsulating common prediction problems in the health domain and allows users to build predictive models with their own data. This system relies on two components: Fast Healthcare Interoperability Resources (FHIR) -- a standardized data structure for electronic health systems -- and several AUTOML frameworks for automated feature engineering, model selection, and tuning. We augment these components with an adaptive data assembler and comprehensive data- and model- auditing capabilities. We demonstrate our framework via 5 prediction tasks on MIMIC-III and Kaggle datasets, which highlight Cardea's human competitiveness, flexibility in problem definition, extensive feature generation capability, adaptable automatic data assembler, and its usability.

In this paper, we propose POTATOES (Partitioning OverfiTting AuTOencoder EnSemble) a new type of autoencoder ensembles for unsupervised outlier detection. Autoencoders are a popular method for this type of problem, especially if the data is located near a submanifold of smaller dimension than that of the ambient space. The standard approach is to approximate the data with the decoder submanifold of the autoencoder and to use the reconstruction error as anomaly score. However, one of the main problems is often to find the right amount of regularization. If the regularization is too strong, the data is underfitted and we obtain many false positives. If the regularization is too weak, the data is overfitted which results in false negatives. The remedy we propose is to not regularize at all, but to rather randomly partition the data into sufficiently many equally sized parts, overfit each part with its own autoencoder, and to use the maximum over all autoencoder reconstruction errors as the anomaly score. We apply our model to realistic data and show that it often outperforms current outlier detection methods.

Energy-based models (EBMs) have recently been successful in representing complex distributions of small images. However, sampling from them requires expensive Markov chain Monte Carlo (MCMC) iterations that mix slowly in high dimensional pixel space. Unlike EBMs, variational autoencoders (VAEs) generate samples quickly and are equipped with a latent space that enables fast traversal of the data manifold. However, VAEs tend to assign high probability density to regions in data space outside the actual data distribution and often fail at generating sharp images. In this paper, we propose VAEBM, a symbiotic composition of a VAE and an EBM that offers the best of both worlds. VAEBM captures the overall mode structure of the data distribution using a state-of-the-art VAE and it relies on its EBM component to explicitly exclude non-data-like regions from the model and refine the image samples. Moreover, the VAE component in VAEBM allows us to speed up MCMC updates by reparameterizing them in the VAE's latent space. Our experimental results show that VAEBM outperforms state-of-the-art VAEs and EBMs in generative quality on several benchmark image datasets by a large margin. It can generate high-quality images as large as 256$\times$256 pixels with short MCMC chains. We also demonstrate that VAEBM provides complete mode coverage and performs well in out-of-distribution detection.

We present a novel view on principal component analysis (PCA) as a competitive game in which each approximate eigenvector is controlled by a player whose goal is to maximize their own utility function. We analyze the properties of this PCA game and the behavior of its gradient based updates. The resulting algorithm--which combines elements from Oja's rule with a generalized Gram-Schmidt orthogonalization--is naturally decentralized and hence parallelizable through message passing. We demonstrate the scalability of the algorithm with experiments on large image datasets and neural network activations. We discuss how this new view of PCA as a differentiable game can lead to further algorithmic developments and insights.

Despite numerous attempts sought to provide empirical evidence of adversarial regularization outperforming sole supervision, the theoretical understanding of such phenomena remains elusive. In this study, we aim to resolve whether adversarial regularization indeed performs better than sole supervision at a fundamental level. To bring this insight into fruition, we study vanishing gradient issue, asymptotic iteration complexity, gradient flow and provable convergence in the context of sole supervision and adversarial regularization. The key ingredient is a theoretical justification supported by empirical evidence of adversarial acceleration in gradient descent. In addition, motivated by a recently introduced unit-wise capacity based generalization bound, we analyze the generalization error in adversarial framework. Guided by our observation, we cast doubts on the ability of this measure to explain generalization. We therefore leave as open questions to explore new measures that can explain generalization behavior in adversarial learning. Furthermore, we observe an intriguing phenomenon in the neural embedded vector space while contrasting adversarial learning with sole supervision.