When deploying machine learning estimators in science and engineering (SAE) domains, it is critical to avoid failed estimations that can have disastrous consequences, e.g., in aero engine design.

When deploying machine learning estimators in science and engineering (SAE) domains, it is critical to avoid failed estimations that can have disastrous consequences, e.g., in aero engine design.

When deploying machine learning estimators in science and engineering (SAE) domains, it is critical to avoid failed estimations that can have disastrous consequences, e.g., in aero engine design. This work focuses on detecting and correcting failed state estimations before adopting them in SAE inverse problems, by utilizing simulations and performance metrics guided by physical laws. We suggest to flag a machine learning estimation when its physical model error exceeds a feasible threshold, and propose a novel approach, GEESE, to correct it through optimization, aiming at delivering both low error and high efficiency. The key designs of GEESE include (1) a hybrid surrogate error model to provide fast error estimations to reduce simulation cost and to enable gradient based backpropagation of error feedback, and (2) two generative models to approximate the probability distributions of the candidate states for simulating the exploitation and exploration behaviours. All three models are constructed as neural networks. GEESE is tested on three real-world SAE inverse problems and compared to a number of state-of-the-art optimization/search approaches. Results show that it fails the least number of times in terms of finding a feasible state correction, and requires physical evaluations less frequently in general.

Deep clustering has been dominated by flat models, which split a dataset into a predefined number of groups. Although recent methods achieve an extremely high similarity with the ground truth on popular benchmarks, the information contained in the flat partition is limited. In this paper, we introduce CoHiClust, a Contrastive Hierarchical Clustering model based on deep neural networks, which can be applied to typical image data. By employing a self-supervised learning approach, CoHiClust distills the base network into a binary tree without access to any labeled data. The hierarchical clustering structure can be used to analyze the relationship between clusters, as well as to measure the similarity between data points. Experiments demonstrate that CoHiClust generates a reasonable structure of clusters, which is consistent with our intuition and image semantics. Moreover, it obtains superior clustering accuracy on most of the image datasets compared to the state-of-the-art flat clustering models. Our implementation is available at https://github.

Current methods for estimating the required neural-network size for a given problem class have focused on methods that can be computationally intensive, such as neural-architecture search and pruning. In contrast, methods that add capacity to neural networks as needed may provide similar results to architecture search and pruning, but do not require as much computation to find an appropriate network size. Here, we present a network-growth method that searches for explainable error in the network's residuals and grows the network if sufficient error is detected. We demonstrate this method using examples from classification, imitation learning, and reinforcement learning. Within these tasks, the growing network can often achieve better performance than small networks that do not grow, and similar performance to networks that begin much larger.

This article describes what neural patches and patch systems are, their advantage over tradition neural network design, and why we're looking for people to train interesting artificial neural patches for image classification. It goes over the steps to train such patches using a simple Windows tool, how to test them in the wild on mobile devices (iOS and Android) and submit them for publication review. In 2006 researchers used fMRI (functional magnetic resonance imaging) and electrical recordings of individual nerve cells to find regions of the inferior temporal lobe that become active when macaque monkeys observe another monkey's face. They found that some nerve regions are triggered only when a face is identified. And those trigger other regions which show sensitivity to only specific orientations of the face, or to specific feature exaggerations. Such regions of a neural network that are conditionally activated in the presence of certain coarse features, and then extract more finer features, are referred to as Neural Patches.

This article describes what neural patches and patch systems are, their advantage over tradition neural network design, and why we're looking for people to train interesting artificial neural patches for image classification. It goes over the steps to train such patches using a simple Windows tool, how to test them in the wild on mobile devices (iOS and Android) and submit them for publication review. In 2006 researchers used fMRI (functional magnetic resonance imaging) and electrical recordings of individual nerve cells to find regions of the inferior temporal lobe that become active when macaque monkeys observe another monkey's face. They found that some nerve regions are triggered only when a face is identified. And those trigger other regions which show sensitivity to only specific orientations of the face, or to specific feature exaggerations. Such regions of a neural network that are conditionally activated in the presence of certain coarse features, and then extract more finer features, are referred to as Neural Patches.

Incrementally training deep neural networks to recognize new classes is a challenging problem. Most existing class-incremental learning methods store data or use generative replay, both of which have drawbacks, while 'rehearsal-free' alternatives such as parameter regularization or bias-correction methods do not consistently achieve high performance. Here, we put forward a new strategy for class-incremental learning: generative classification. Rather than directly learning the conditional distribution p(y|x), our proposal is to learn the joint distribution p(x,y), factorized as p(x|y)p(y), and to perform classification using Bayes' rule. As a proof-of-principle, here we implement this strategy by training a variational autoencoder for each class to be learned and by using importance sampling to estimate the likelihoods p(x|y). This simple approach performs very well on a diverse set of continual learning benchmarks, outperforming generative replay and other existing baselines that do not store data.