Deep neural networks trained using a softmax layer at the top and the cross-entropy loss are ubiquitous tools for image classification. Yet, this does not naturally enforce intra-class similarity nor inter-class margin of the learned deep representations. To simultaneously achieve these two goals, different solutions have been proposed in the literature, such as the pairwise or triplet losses. However, such solutions carry the extra task of selecting pairs or triplets, and the extra computational burden of computing and learning for many combinations of them. In this paper, we propose a plug-and-play loss term for deep networks that explicitly reduces intra-class variance and enforces inter-class margin simultaneously, in a simple and elegant geometric manner. For each class, the deep features are collapsed into a learned linear subspace, or union of them, and inter-class subspaces are pushed to be as orthogonal as possible. Our proposed Orthogonal Low-rank Embedding (OL\'E) does not require carefully crafting pairs or triplets of samples for training, and works standalone as a classification loss, being the first reported deep metric learning framework of its kind. Because of the improved margin between features of different classes, the resulting deep networks generalize better, are more discriminative, and more robust. We demonstrate improved classification performance in general object recognition, plugging the proposed loss term into existing off-the-shelf architectures. In particular, we show the advantage of the proposed loss in the small data/model scenario, and we significantly advance the state-of-the-art on the Stanford STL-10 benchmark.

Deep generative models have been successfully used to learn representations for high-dimensional discrete spaces by representing discrete objects as sequences, for which powerful sequence-based deep models can be employed. Unfortunately, these techniques are significantly hindered by the fact that these generative models often produce invalid sequences: sequences which do not represent any underlying discrete structure. As a step towards solving this problem, we propose to learn a deep recurrent validator model, which can estimate whether a partial sequence can function as the beginning of a full, valid sequence. This model not only discriminates between valid and invalid sequences, but also provides insight as to how individual sequence elements influence the validity of the overall sequence, and the existence of a corresponding discrete object. To learn this model we propose a reinforcement learning approach, where an oracle which can evaluate validity of complete sequences provides a sparse reward signal. We believe this is a key step toward learning generative models that faithfully produce valid sequences which represent discrete objects. We demonstrate its effectiveness in evaluating the validity of Python 3 source code for mathematical expressions, and improving the ability of a variational autoencoder trained on SMILES strings to decode valid molecular structures.

The computer-aided analysis of medical scans is a longstanding goal in the medical imaging field. Currently, deep learning has became a dominant methodology for supporting pathologists and radiologist. Deep learning algorithms have been successfully applied to digital pathology and radiology, nevertheless, there are still practical issues that prevent these tools to be widely used in practice. The main obstacles are low number of available cases and large size of images (a.k.a. the small n, large p problem in machine learning), and a very limited access to annotation at a pixel level that can lead to severe overfitting and large computational requirements. We propose to handle these issues by introducing a framework that processes a medical image as a collection of small patches using a single, shared neural network. The final diagnosis is provided by combining scores of individual patches using a permutation-invariant operator (combination). In machine learning community such approach is called a multi-instance learning (MIL).

Hyperparameter optimization undergoes extensive evaluations of validation errors in order to find its best configuration. Bayesian optimization is now popular for hyperparameter optimization, since it reduces the number of validation error evaluations required. Suppose that we are given a collection of datasets on which hyperparameters are already tuned by either humans with domain expertise or extensive trials of cross-validation. When a model is applied to a new dataset, it is desirable to let Bayesian optimization start from configurations that were successful on similar datasets. To this end, we construct a Siamese network with convolutional layers followed by bi-directional LSTM layers, to learn meta-features over image datasets. Learned meta-features are used to select a few datasets that are similar to the new dataset, so that a set of configurations in similar datasets is adopted as initialization to warm-start Bayesian hyperparameter optimization. Experiments on image datasets demonstrate that our learned meta-features are useful in optimizing hyperparameters in deep residual networks for image classification.

Machine Learning (ML) has found it particularly useful in malware detection. However, as the malware evolves very fast, the stability of the feature extracted from malware serves as a critical issue in malware detection. Recent success of deep learning in image recognition, natural language processing, and machine translation indicate a potential solution for stabilizing the malware detection effectiveness. We present a coloR-inspired convolutional neuRal network-based AndroiD malware Detection (R2-D2), which can detect malware without extracting pre-selected features (e.g., the control-flow of op-code, classes, methods of functions and the timing they are invoked etc.) from Android apps. In particular, we develop a color representation for translating Android apps into RGB color code and transform them to a fixed-sized encoded image. After that, the encoded image is fed to convolutional neural network for automatic feature extraction and learning, reducing the expert's intervention. We have collected over 1 million malware samples and 1 million benign samples according to the data provided by Leopard Mobile Inc. from its core product Security Master (which has 623 million monthly active users and 10k new malware samples per day). It is shown that R2-D2 can effectively detect the malware. Furthermore, we keep our research results and release experiment material on http://R2D2.TWMAN.ORG if there is any update.

Collected from the English exams for middle and high school Chinese students in the age range between 12 to 18, RACE consists of near 28,000 passages and near 100,000 questions generated by human experts (English instructors), and covers a variety of topics which are carefully designed for evaluating the students' ability in understanding and reasoning. In particular, the proportion of questions that requires reasoning is much larger in RACE than that in other benchmark datasets for reading comprehension, and there is a significant gap between the performance of the state-of-the-art models (43%) and the ceiling human performance (95%). We hope this new dataset can serve as a valuable resource for research and evaluation in machine comprehension.

Which authors of this paper are endorsers? Disable MathJax (What is MathJax?)

IMAGE: This is an illustration of a multi-compartment neural network model for deep learning. Right: Illustration of simplified pyramidal neuron models.... view more Deep learning has brought about machines that can'see' the world more like humans can, and recognize language. And while deep learning was inspired by the human brain, the question remains: Does the brain actually learn this way? The answer has the potential to create more powerful artificial intelligence and unlock the mysteries of human intelligence. In a study published December 5th in eLife, CIFAR Fellow Blake Richards and his colleagues unveiled an algorithm that simulates how deep learning could work in our brains.

Prateek Joshi is an artificial intelligence researcher, published author of five books, and TEDx speaker. He is the founder of Pluto AI, a venture-funded Silicon Valley startup building an analytics platform for smart water management powered by deep learning. His work in this field has led to patents, tech demos, and research papers at major IEEE conferences. He has been an invited speaker at technology and entrepreneurship conferences including TEDx, AT&T Foundry, Silicon Valley Deep Learning, and Open Silicon Valley. Prateek has also been featured as a guest author in prominent tech magazines.

Artificial intelligence in healthcare is at the end of the beginning: It's been researched, introduced, proven to work, and it's been put to use in real clinical settings. Healthcare has only explored the tip of the iceberg and, of course, there is much work ahead to improve patient care. AI in healthcare is beginning to emerge out of its infancy, said Ted Willke, senior principal engineer at Intel Labs. "We're seeing healthcare organizations and hospitals move beyond AI-based proof-of-concepts and program pilots into developing and adopting systems that work the best for their needs," Willke said. "AI in healthcare, like in other industries, began as a way to help these organizations manage their vast amounts of data and simplify daily tasks, but we're starting to see the emergence of truly innovative uses of AI in healthcare – from finding complex patterns in medical imaging to genomic sequencing to designing patient treatment plans."