Semi-supervised learning algorithms reduce the high cost of acquiring labeled training data by using both labeled and unlabeled data during learning. Deep Convolutional Networks (DCNs) have achieved great success in supervised tasks and as such have been widely employed in the semi-supervised learning. In this paper we leverage the recently developed Deep Rendering Mixture Model (DRMM), a probabilistic generative model that models latent nuisance variation, and whose inference algorithm yields DCNs. We develop an EM algorithm for the DRMM to learn from both labeled and unlabeled data. Guided by the theory of the DRMM, we introduce a novel non-negativity constraint and a variational inference term. We report state-of-the-art performance on MNIST and SVHN and competitive results on CIFAR10. We also probe deeper into how a DRMM trained in a semi-supervised setting represents latent nuisance variation using synthetically rendered images. Taken together, our work provides a unified framework for supervised, unsupervised, and semi-supervised learning.

We present a general theoretical analysis of structured prediction with a series of new results. We give new data-dependent margin guarantees for structured prediction for a very wide family of loss functions and a general family of hypotheses, with an arbitrary factor graph decomposition. These are the tightest margin bounds known for both standard multi-class and general structured prediction problems. Our guarantees are expressed in terms of a data-dependent complexity measure, factor graph complexity, which we show can be estimated from data and bounded in terms of familiar quantities. We further extend our theory by leveraging the principle of Voted Risk Minimization (VRM) and show that learning is possible even with complex factor graphs. We present new learning bounds for this advanced setting, which we use to design two new algorithms, Voted Conditional Random Field (VCRF) and Voted Structured Boosting (StructBoost). These algorithms can make use of complex features and factor graphs and yet benefit from favorable learning guarantees. We also report the results of experiments with VCRF on several datasets to validate our theory.

Many inference problems in structured prediction are naturally solved by augmenting a tractable dependency structure with complex, non-local auxiliary objectives. This includes the mean field family of variational inference algorithms, soft- or hard-constrained inference using Lagrangian relaxation or linear programming, collective graphical models, and forms of semi-supervised learning such as posterior regularization. We present a method to discriminatively learn broad families of inference objectives, capturing powerful non-local statistics of the latent variables, while maintaining tractable and provably fast inference using non-Euclidean projected gradient descent with a distance-generating function given by the Bethe entropy. We demonstrate the performance and flexibility of our method by (1) extracting structured citations from research papers by learning soft global constraints, (2) achieving state-of-the-art results on a widely-used handwriting recognition task using a novel learned non-convex inference procedure, and (3) providing a fast and highly scalable algorithm for the challenging problem of inference in a collective graphical model applied to bird migration.

Functional MRI (fMRI) and diffusion MRI (dMRI) are non-invasive imaging modalities that allow in-vivo analysis of a patient's brain network (known as a connectome). Use of these technologies has enabled faster and better diagnoses and treatments of neurological disorders and a deeper understanding of the human brain. Recently, researchers have been exploring the application of machine learning models to connectome data in order to predict clinical outcomes and analyze the importance of subnetworks in the brain. Connectome data has unique properties, which present both special challenges and opportunities when used for machine learning. The purpose of this work is to review the literature on the topic of applying machine learning models to MRI-based connectome data. This field is growing rapidly and now encompasses a large body of research. To summarize the research done to date, we provide a comparative, structured summary of 77 relevant works, tabulated according to different criteria, that represent the majority of the literature on this topic. (We also published a living version of this table online at http://connectomelearning.cs.sfu.ca that the community can continue to contribute to.) After giving an overview of how connectomes are constructed from dMRI and fMRI data, we discuss the variety of machine learning tasks that have been explored with connectome data. We then compare the advantages and drawbacks of different machine learning approaches that have been employed, discussing different feature selection and feature extraction schemes, as well as the learning models and regularization penalties themselves. Throughout this discussion, we focus particularly on how the methods are adapted to the unique nature of graphical connectome data. Finally, we conclude by summarizing the current state of the art and by outlining what we believe are strategic directions for future research.

Hundreds of the nation's top female soccer players are expected to gather in Norco on Friday for the first day of a three-day college showcase. More than 140 registered teams from all over the western U.S. are scheduled to compete before more than 100 coaches from 16 conferences and more than three dozen states. Among the elite clubs who have confirmed their participation are Slammers FC, Legends FC, Eagles SC, Sereno Soccer Club of Arizona, LA Premier FC and Pateadores SC. The event kicks off at 8 a.m. For information, go to the tournament's website at silverlakestournaments.com.

This paper presents an infinite variational autoencoder (VAE) whose capacity adapts to suit the input data. This is achieved using a mixture model where the mixing coefficients are modeled by a Dirichlet process, allowing us to integrate over the coefficients when performing inference. Critically, this then allows us to automatically vary the number of autoencoders in the mixture based on the data. Experiments show the flexibility of our method, particularly for semi-supervised learning, where only a small number of training samples are available.

What do you do when you're reading an article or paper, one that it's very important you understand, and get stumped by a particular passage? More often than not, you'll head over to Google--or whatever your favorite search engine is--start surfing the Web, and won't stop until you find a satisfactory answer to the puzzle. Researchers at MIT have developed a machine learning system that behaves much the same way in the course of performing information extraction, the process of creating structured data from unstructured formats such as plain text. Here are the key details from MIT's newsroom: Most machine-learning systems work by combing through training examples and looking for patterns that correspond to classifications provided by human annotators. For instance, humans might label parts of speech in a set of texts, and the machine-learning system will try to identify patterns that resolve ambiguities -- for instance, when "her" is a direct object and when it's an adjective.

Machine learning is basically teaching computers to solve big problems based on either example data or past experiences. Example data, is purely unlabeled, with unknown and undetected structure. Your power would rely on you guessing the hidden structure which ultimately leads in you learning more about it. Using technical terminologies, unsupervised learning best describes the latter. Past experiences on the other hand, is real data with clear labels and answers to the question you are trying to answer.

Researchers from the US have developed an artificial intelligence (AI) system that surfs the internet, extracts information from the available plain text and organizes it for quantitative analysis in very less time. Recently at the Association for Computational Linguistics' Conference on Empirical Methods on Natural Language Processing, researchers from the Massachusetts Institute of Technology (MIT) Computer Science and Artificial Intelligence Laboratory won a best-paper award for a new approach to information extraction that turns conventional machine learning on its head. Most machine-learning systems work by combing through training examples and looking for patterns that correspond to classifications provided by human annotators. In their new paper, the MIT researchers trained their system on scanty data -- because in the scenario they're investigating, that's usually all that's available. But then they find the limited information an easy problem to solve.

Scientists have developed a new artificial intelligence system that can more effectively extract data from the vast wealth of information present on the internet. The data necessary to answer myriad questions - about, say, the correlations between the industrial use of certain chemicals and incidents of disease, or between patterns of news coverage and voter-poll results - may all be online in form of plain text. However, extracting data from plain text and organising it for quantitative analysis may be prohibitively time consuming. Researchers from Massachusetts Institute of Technology (MIT) in the US developed a new approach to information extraction. Most machine-learning systems work by combing through training examples and looking for patterns that correspond to classifications provided by human annotators.