Most state-of-the-art Deep Learning (DL) approaches for speaker recognition work on a short utterance level. Given the speech signal, these algorithms extract a sequence of speaker embeddings from short segments and those are averaged to obtain an utterance level speaker representation. In this work we propose the use of an attention mechanism to obtain a discriminative speaker embedding given non fixed length speech utterances. Our system is based on a Convolutional Neural Network (CNN) that encodes short-term speaker features from the spectrogram and a self multi-head attention model that maps these representations into a long-term speaker embedding. The attention model that we propose produces multiple alignments from different subsegments of the CNN encoded states over the sequence. Hence this mechanism works as a pooling layer which decides the most discriminative features over the sequence to obtain an utterance level representation. We have tested this approach for the verification task for the VoxCeleb1 dataset. The results show that self multi-head attention outperforms both temporal and statistical pooling methods with a 18\% of relative EER. Obtained results show a 58\% relative improvement in EER compared to i-vector+PLDA.

Despite their great success in recent years, deep neural networks (DNN) are mainly black boxes where the results obtained by running through the network are difficult to understand and interpret. Compared to e.g. decision trees or bayesian classifiers, DNN suffer from bad interpretability where we understand by interpretability, that a human can easily derive the relations modeled by the network. A reasonable way to provide interpretability for humans are logical rules. In this paper we propose neural logic rule layers (NLRL) which are able to represent arbitrary logic rules in terms of their conjunctive and disjunctive normal forms. Using various NLRL within one layer and correspondingly stacking various layers, we are able to represent arbitrary complex rules by the resulting neural network architecture. The NLRL are end-to-end trainable allowing to learn logic rules directly from available data sets. Experiments show that NLRL-enhanced neural networks can learn to model arbitrary complex logic and perform arithmetic operation over the input values.

We introduce a modular system that can be deployed on any Kubernetes cluster for question answering via REST API. This system, called Katecheo, includes four configurable modules that collectively enable identification of questions, classification of those questions into topics, a search of knowledge base articles, and reading comprehension. We demonstrate the system using publicly available, pre-trained models and knowledge base articles extracted from Stack Exchange sites. However, users can extend the system to any number of topics, or domains, without the need to modify any of the model serving code. All components of the system are open source and available under a permissive Apache 2 License.

There is renewed interest in simulating language emergence among deep neural agents that communicate to jointly solve a task, spurred by the practical aim to develop language-enabled interactive AIs, as well as by theoretical questions about the evolution of human language. However, optimizing deep architectures connected by a discrete communication channel (such as that in which language emerges) is technically challenging. We introduce EGG, a toolkit that greatly simplifies the implementation of emergent-language communication games. EGG's modular design provides a set of building blocks that the user can combine to create new games, easily navigating the optimization and architecture space. We hope that the tool will lower the technical barrier, and encourage researchers from various backgrounds to do original work in this exciting area.

Developing agents that can quickly adapt their behavior to new tasks remains a challenge. Meta-learning has been applied to this problem, but previous methods require either specifying a reward function which can be tedious or providing demonstrations which can be inefficient. In this paper, we investigate if, and how, a "helper" agent can be trained to interactively adapt their behavior to maximize the reward of another agent, whom we call the "prime" agent, without observing their reward or receiving explicit demonstrations. To this end, we propose to meta-learn a helper agent along with a prime agent, who, during training, observes the reward function and serves as a surrogate for a human prime. We introduce a distribution of multi-agent cooperative foraging tasks, in which only the prime agent knows the objects that should be collected. We demonstrate that, from the emerged physical communication, the trained helper rapidly infers and collects the correct objects.

In this article, I will present several techniques for you to make your first steps towards developing an algorithm that could be used for a classic image classification problem: detecting dog breed from an image. By the end of this article, we'll have developed code that will accept any user-supplied image as input and return an estimate of the dog's breed. Also, if a human is detected, the algorithm will provide an estimate of the dog breed that is most resembling. This project was completed as part of Udacity's Machine Learning Nanodegree (GitHub repo). Convolutional neural networks (also refered to as CNN or ConvNet) are a class of deep neural networks that have seen widespread adoption in a number of computer vision and visual imagery applications.

The world of computing, from chips to software to systems, is going to change dramatically in coming years as a result of the spread of machine learning. We may still refer to these computers as "Universal Turing Machines," as we have for eighty years or more. But in practice they will be different from the way they have been built and used up to now. Such a change is of interest both to anyone who cares about what computers do, and to anyone who's interested in machine learning in all its forms. In February, Facebook's head of A.I. research, Yann LeCun, gave a talk at the International Solid State Circuits Conference in San Francisco, one of the longest running computer chip conferences in the world.

The 2nd International MIDL Conference on Medical Imaging with Deep Learning brings deep learning and medical imaging researchers together for in-depth discussion and exchange of ideas. This conference will be a forum for deep learning researchers, clinicians and healthcare companies to take a leap in the application of deep learning based automatic image analysis in disease screening, diagnosis, prognosis, treatment selection and treatment monitoring. The conference will have a broad scope including topics such as computer-aided screening and diagnosis, intervention, detection, segmentation, registration, image reconstruction and synthesis etc..

Effect of Population Based Augmentation applied to images, which differs at different percentages into training. In this blog post we introduce Population Based Augmentation (PBA), an algorithm that quickly and efficiently learns a state-of-the-art approach to augmenting data for neural network training. PBA matches the previous best result on CIFAR and SVHN but uses one thousand times less compute, enabling researchers and practitioners to effectively learn new augmentation policies using a single workstation GPU. You can use PBA broadly to improve deep learning performance on image recognition tasks. We discuss the PBA results from our recent paper and then show how to easily run PBA for yourself on a new data set in the Tune framework.

At last week's Conference on Computer Vision and Pattern Recognition, a team of researchers from Massachusetts Institute of Technology (MIT) presented an innovative artificial intelligence (AI) system that can learn to segment anatomical brain structures from a single segmented brain scan image along with unlabeled scans--automating neuroscientific image segmentation. This novel AI system for neuroscience originated from a very distant genre of smartphone and gaming. Amy Zhao, a graduate student in the Department of Electrical Engineering and Computer Science (EECS) and Computer Science and Artificial Intelligence Laboratory (CSAIL), and the first author on the research, initially sought to create an app using convolutional neural network technology that could provide detailed information in real-time about cards from the game "Magic: The Gathering" based on a picture taken on a smartphone. The challenge is that this computer vision task would require a data set of photos that contains not only each of the 20,000 cards, but also many more images of each card with variation in appearance and attributes such as lighting. Creating such a data set manually would be painstakingly tedious and extremely time-consuming.