This paper introduces a machine for sampling approximate model-X knockoffs for arbitrary and unspecified data distributions using deep generative models. The main idea is to iteratively refine a knockoff sampling mechanism until a criterion measuring the validity of the produced knockoffs is optimized; this criterion is inspired by the popular maximum mean discrepancy in machine learning and can be thought of as measuring the distance to pairwise exchangeability between original and knockoff features. By building upon the existing model-X framework, we thus obtain a flexible and model-free statistical tool to perform controlled variable selection. Extensive numerical experiments and quantitative tests confirm the generality, effectiveness, and power of our deep knockoff machines. Finally, we apply this new method to a real study of mutations linked to changes in drug resistance in the human immunodeficiency virus.

Making an informed, correct and quick decision can be life-saving. It's crucial for animals during an escape behaviour or for autonomous cars during driving. The decision can be complex and may involve an assessment of the amount of threats present and the nature of each threat. Thus, we should expect early sensory processing to supply classification information fast and accurately, even before relying the information to higher brain areas or more complex system components downstream. Today, advanced convolutional artificial neural networks can successfully solve visual detection and classification tasks and are commonly used to build complex decision making systems. However, in order to perform well on these tasks they require increasingly complex, "very deep" model structure, which is costly in inference run-time, energy consumption and number of training samples, only trainable on cloud-computing clusters. A single spiking neuron has been shown to be able to solve recognition tasks for homogeneous Poisson input statistics, a commonly used model for spiking activity in the neocortex. When modeled as leaky integrate and fire with gradient decent learning algorithm it was shown to posses a variety of complex computational capabilities. Here we improve its implementation. We also account for more natural stimulus generated inputs that deviate from this homogeneous Poisson spiking. The improved gradient-based local learning rule allows for significantly better and stable generalization. We also show that with its improved capabilities it can count weakly labeled concepts by applying our model to a problem of multiple instance learning (MIL) with counting where labels are only available for collections of concepts. In this counting MNIST task the neuron exploits the improved implementation and outperforms conventional ConvNet architecture under similar condtions.

Getting pain intensity from face images is an important problem in autonomous nursing systems. However, due to the limitation in data sources and the subjectiveness in pain intensity values, it is hard to adopt modern deep neural networks for this problem without domain-specific auxiliary design. Inspired by human vision priori, we propose a novel approach called saliency supervision, where we directly regularize deep networks to focus on facial area that is discriminative for pain regression. Through alternative training between saliency supervision and global loss, our method can learn sparse and robust features, which is proved helpful for pain intensity regression. We verified saliency supervision with face-verification network backbone on the widely-used dataset, and achieved state-of-art performance without bells and whistles. Our saliency supervision is intuitive in spirit, yet effective in performance. We believe such saliency supervision is essential in dealing with ill-posed datasets, and has potential in a wide range of vision tasks.

Affect conveys important implicit information in human communication. Having the capability to correctly express affect during human-machine conversations is one of the major milestones in artificial intelligence. In recent years, extensive research on open-domain neural conversational models has been conducted. However, embedding affect into such models is still under explored. In this paper, we propose an end-to-end affect-rich open-domain neural conversational model that produces responses not only appropriate in syntax and semantics, but also with rich affect. Our model extends the Seq2Seq model and adopts VAD (Valence, Arousal and Dominance) affective notations to embed each word with affects. In addition, our model considers the effect of negators and intensifiers via a novel affective attention mechanism, which biases attention towards affect-rich words in input sentences. Lastly, we train our model with an affect-incorporated objective function to encourage the generation of affect-rich words in the output responses. Evaluations based on both perplexity and human evaluations show that our model outperforms the state-of-the-art baseline model of comparable size in producing natural and affect-rich responses.

The increasing concern with misinformation has stimulated research efforts on automatic fact checking. The recently-released FEVER dataset introduced a benchmark fact-verification task in which a system is asked to verify a claim using evidential sentences from Wikipedia documents. In this paper, we present a connected system consisting of three homogeneous neural semantic matching models that conduct document retrieval, sentence selection, and claim verification jointly for fact extraction and verification. For evidence retrieval (document retrieval and sentence selection), unlike traditional vector space IR models in which queries and sources are matched in some pre-designed term vector space, we develop neural models to perform deep semantic matching from raw textual input, assuming no intermediate term representation and no access to structured external knowledge bases. We also show that Pageview frequency can also help improve the performance of evidence retrieval results, that later can be matched by using our neural semantic matching network. For claim verification, unlike previous approaches that simply feed upstream retrieved evidence and the claim to a natural language inference (NLI) model, we further enhance the NLI model by providing it with internal semantic relatedness scores (hence integrating it with the evidence retrieval modules) and ontological WordNet features. Experiments on the FEVER dataset indicate that (1) our neural semantic matching method outperforms popular TF-IDF and encoder models, by significant margins on all evidence retrieval metrics, (2) the additional relatedness score and WordNet features improve the NLI model via better semantic awareness, and (3) by formalizing all three subtasks as a similar semantic matching problem and improving on all three stages, the complete model is able to achieve the state-of-the-art results on the FEVER test set.

Lifelong Machine Learning considers systems that can learn many tasks (from one or more domains) over a lifetime (Thrun, 1998; Silver et al., 2013). This has several names and manifestations in the literature: incremental learning (Solomonoff, 1989), continual learning (Ring, 1997), explanation-based learning (Thrun, 1996, 2012), never ending learning (Carlson et al., 2010), etc. The underlying idea motivating these efforts is the following: Lifelong learning systems would be more effective at learning and retaining knowledge across different tasks. By using the prior knowledge and exploiting similarity acrosstasks, they would be able to obtain better priors for the task at hand. Lifelong learning techniques are very important for training intelligent autonomous agents that would need to operate and make decisions over extended periods of time. These characteristics arespecially important in the industrial setups where the deployed machine learning models are being updated frequently with new incoming data whose distribution neednot match the data on which the model was originally trained. Lifelong learning is an extremely challenging task for the machine learning models because of two primary reasons: 1. Catastrophic Forgetting: As the model is trained on a new task (or a new data/task distribution), it is likely to forget the knowledge it acquired from the previous tasks (or data distributions). This phenomenon is also known as the catastrophic interference (McCloskey and Cohen, 1989).

A Google researcher has highlighted some of the hilarious ways that artificial intelligence (AI) software has'cheated' to fulfil its purpose. A programme designed not to lose at Tetris completed its task by simply pausing the game, while a self-driving car simulator asked to keep cars'fast and safe' did so by making them spin on the spot. An AI programmed to spot cancerous skin lesions learned to flag blemishes pictured next to a ruler, as they indicated humans were already concerned about them. Victoria Krakovna, of Google's DeepMind AI lab, asked her colleagues for examples of misbehaving AI to highlight an often overlooked danger of the technology. She said that the biggest threat posed by AI was not that they disobeyed us, but that they obeyed us in the wrong way.

To the general public, today's "AI" technologies are nothing short of magic. Algorithms that can eerily understand video, images, speech, and text, translate between languages with uncanny accuracy, drive cars, play video games, find cancer and even best humans at complex strategy games by developing novel moves that no human had ever devised. Groundbreaking new milestones are crossed almost daily. Computer science programs are flooded with students eager to become AI experts and companies can't hire enough AI programmers. It would seem the era of AI has truly arrived.

In this tutorial, you'll get an introduction to deep learning using the PyTorch framework, and by its conclusion, you'll be comfortable applying it to your deep learning models. Facebook launched PyTorch 1.0 early this year with integrations for Google Cloud, AWS, and Azure Machine Learning. In this tutorial, I assume that you're already familiar with Scikit-learn, Pandas, NumPy, and SciPy. These packages are important prerequisites for this tutorial. Deep learning is a subfield of machine learning with algorithms inspired by the working of the human brain.

Material irradiation experiment is dangerous and complex, thus it requires those with a vast advanced expertise to process the images and data manually. In this paper, we propose a generative adversarial model based on prior knowledge and attention mechanism to achieve the generation of irradiated material images (data-to-image model), and a prediction model for corresponding industrial performance (image-to-data model). With the proposed models, researchers can skip the dangerous and complex irradiation experiments and obtain the irradiation images and industrial performance parameters directly by inputing some experimental parameters only. We also introduce a new dataset ISMD which contains 22000 irradiated images with 22,143 sets of corresponding parameters. Our model achieved high quality results by compared with several baseline models. The evaluation and detailed analysis are also performed.