Prediction models based on deep neural networks are increasingly gaining attention for fast and accurate virtual screening systems. For decision makings in virtual screening, researchers find it useful to interpret an output of classification system as probability, since such interpretation allows them to filter out more desirable compounds. However, probabilistic interpretation cannot be correct for models that hold over-parameterization problems or inappropriate regularizations, leading to unreliable prediction and decision making. In this regard, we concern the reliability of neural prediction models on molecular properties, especially when models are trained with sparse data points and imbalanced distributions. This work aims to propose guidelines for training reliable models, we thus provide methodological details and ablation studies on the following train principles. We investigate the effects of model architectures, regularization methods, and loss functions on the prediction performance and reliability of classification results. Moreover, we evaluate prediction reliability of models on virtual screening scenario. Our result highlights that correct choice of regularization and inference methods is evidently important to achieve high success rate, especially in data imbalanced situation. All experiments were performed under a single unified model implementation to alleviate external randomness in model training and to enable precise comparison of results.

Network quantization has rapidly become one of the most widely used methods to compress and accelerate deep neural networks. Recent efforts propose to quantize weights and activations from different layers with different precision to improve the overall performance. However, it is challenging to find the optimal bitwidth (i.e., precision) for weights and activations of each layer efficiently. Meanwhile, it is yet unclear how to perform convolution for weights and activations of different precision efficiently on generic hardware platforms. To resolve these two issues, in this paper, we first propose an Efficient Bitwidth Search (EBS) algorithm, which reuses the meta weights for different quantization bitwidth and thus the strength for each candidate precision can be optimized directly w.r.t the objective without superfluous copies, reducing both the memory and computational cost significantly. Second, we propose a binary decomposition algorithm that converts weights and activations of different precision into binary matrices to make the mixed precision convolution efficient and practical. Experiment results on CI-FAR10 and ImageNet datasets demonstrate our mixed precision QNN outperforms the handcrafted uniform bitwidth counterparts and other mixed precision techniques.

Many machine learning problems consider data where each instance is, itself, an unordered set of elements; i.e., such that each observation is a set. Data of this kind arises in a variety of applications, ranging from document modeling (Blei et al., 2003; Garnelo et al., 2018a) and multi-task learning (Zaheer et al., 2017; Edwards & Storkey, 2016; Liu et al., 2019) to 3D point cloud modeling (Li et al., 2018; Yang et al., 2019). In unsupervised settings, a dataset typically consists of a set of such sets, while in supervised learning, it consists of a set of (set, label) pairs. Modeling a distribution over a space of instances, where each instance is, itself, an unordered set of elements involves two key considerations: (1) the elements within a single instance are exchangeable, i.e., the elements are order invariant; and (2) the cardinalities of the instances (sets) vary, i.e., instances need not exhibit the same cardinality. Modeling both unconditional and conditional distributions over instances (sets) are relevant to consider, since these support unsupervised and supervised tasks respectively. For unconditional distribution modeling, there has been significant prior work on modeling set distributions, which has sought to balance competing needs to expand model flexibility and preserve tractability on the one hand, with respecting exchangeability and varying instance cardinalities on the other hand. However, managing these tradeoffs has proved to be quite difficult, and current approaches remain limited in different respects. For example, a particularly straightforward strategy for modeling distributions over instances x {x 1,..., x n } without assuming fixed cardinality is simply to use a recurrent neural network (RNNs) to encode instance probability auto-regressively via p (x) n

Artificial Intelligence has historically relied on planning, heuristics, and handcrafted approaches designed by experts. All the while claiming to pursue the creation of Intelligence. This approach fails to acknowledge that intelligence emerges from the dynamics within a complex system. Neurons in the brain are governed by local rules, where no single neuron, or group of neurons, coordinates or controls the others. This local structure gives rise to the appropriate dynamics in which intelligence can emerge. Populations of neurons must compete with their neighbors for resources, inhibition, and activity representation. At the same time, they must cooperate, so the population and organism can perform high-level functions. To this end, we introduce modeling neurons as reinforcement learning agents. Where each neuron may be viewed as an independent actor, trying to maximize its own self-interest. By framing learning in this way, we open the door to an entirely new approach to building intelligent systems.

With the broader and highly successful usage of machine learning in industry and the sciences, there has been a growing demand for explainable AI. Interpretability and explanation methods for gaining a better understanding about the problem solving abilities and strategies of nonlinear Machine Learning such as Deep Learning (DL), LSTMs, and kernel methods are therefore receiving increased attention. In this work we aim to (1) provide a timely overview of this active emerging field and explain its theoretical foundations, (2) put interpretability algorithms to a test both from a theory and comparative evaluation perspective using extensive simulations, (3) outline best practice aspects i.e. how to best include interpretation methods into the standard usage of machine learning and (4) demonstrate successful usage of explainable AI in a representative selection of application scenarios. Finally, we discuss challenges and possible future directions of this exciting foundational field of machine learning.

Explainability has been an important goal since the early days of Artificial Intelligence. Several approaches for producing explanations have been developed. However, many of these approaches were tightly coupled with the capabilities of the artificial intelligence systems at the time. With the proliferation of AI-enabled systems in sometimes critical settings, there is a need for them to be explainable to end-users and decision-makers. We present a historical overview of explainable artificial intelligence systems, with a focus on knowledge-enabled systems, spanning the expert systems, cognitive assistants, semantic applications, and machine learning domains. Additionally, borrowing from the strengths of past approaches and identifying gaps needed to make explanations user- and context-focused, we propose new definitions for explanations and explainable knowledge-enabled systems.

A standard method for testing some of the mechanical properties of materials is to poke them with a sharp point. This "indentation technique" can provide detailed measurements of how the material responds to the point's force, as a function of its penetration depth. With advances in nanotechnology during the past two decades, the indentation force can be measured to a resolution on the order of one-billionth of a Newton (a measure of the force approximately equivalent to the force you feel when you hold a medium-sized apple in your hand), and the sharp tip's penetration depth can be captured to a resolution as small as a nanometer, or about 1/100,000 the diameter of a human hair. Such instrumented nanoindentation tools have provided new opportunities for probing physical properties in a wide variety of materials, including metals and alloys, plastics, ceramics, and semiconductors. But while indentation techniques, including nanoindentation, work well for measuring some properties, they exhibit large errors when probing plastic properties of materials -- the kind of permanent deformation that happens, for example, if you press your thumb into a piece of silly putty and leave a dent, or when you permanently bend a paper clip using your fingers.

AI for sales is nothing new. But what are the real benefits of AI? Is it a lot of buzz or does it deliver real value? Sales tools using machine learning and deep learning are already widespread in the market today. And while the scope of AI's impact may still be limited to certain types of activities, the power and scope of AI will continue to grow.

Hi. My name is Gabriel Ha, and I'm here to show you how MATLAB makes it straightforward to create a deep neural network from scratch. Our demo has specific application-to-image processing and recognition, but we feel like images are pretty easy to relate to. Most importantly, we want to make deep learning accessible to everyone, and you'll be able to get your hands on everything that we show you, and build on them, and start using your own networks on your own. So for those of you who are super familiar with training networks, along with techniques to make them more accurate, MATLAB is going to be great for you because, as you would expect, we provide you with the intuitive syntax and functions which will allow you to easily implement your refinements. For those of you new to the deep learning field and want to get your feet wet with this technology, the extent of what you can immediately do might be limited to image recognition, but I'm confident that it will provide you with more than enough material to get started and have a lot of fun with neural networks.

We have a PhD level or Advanced courses thread in the sidebar but it's three year old now. There were two other 7-8 month old threads (1, 2) but they don't have many quality responses either. So, can we have a new one here? To reiterate - CS231n, CS229, ones from Udemy etc are not advanced. Advanced ML/DL/RL, attempts at building theory of DL, optimization theory, advanced applications etc are some examples of what I believe should belong here, much like the original sidebar post.