Given a data sample with its feature vector, SC tries to learn a codebook with some codeworks, and approximate the data sample as the linear combination of the codewords. SC assume that only a few codewords in the codebook are enough to represent the data sample, thus the combination coefficients should be sparse, i.e. most of the coefficients are zeros, leaving only a few of them non-zeros. The linear combination coefficients of the data sample could be its new representation. Because they are sparse, the coefficient vector is often referred to as the sparse code. To solve the sparse code, one usually minimizes the approximation error with regard to the codebook and the sparse code, and at the same time seeks the sparsity of the sparse code. Although SC has been used in many pattern recognition applications, such as palmprint recognition [24], dynamic texture recognition [25], human action recognition [26], [27], [28], speech recognition [29], digit recognition [30], image annotation [31], [32], [33], and face recognition [34], in most cases, SC is used as an unsupervised learning method. When SC is performed to the training data set, it is assumed that the class labels of the training samples are unavailable. Then after the sparse codes are learned, they will be used to learn a classifier. Thus the class labels are ignored during the sparse coding procedure.

Identifying and understanding modular organizations is centrally important in the study of complex systems. Several approaches to this problem have been advanced, many framed in information-theoretic terms. Our treatment starts from the complementary point of view of statistical modeling and prediction of dynamical systems. It is known that for finite amounts of training data, simpler models can have greater predictive power than more complex ones. We use the trade-off between model simplicity and predictive accuracy to generate optimal multiscale decompositions of dynamical networks into weakly-coupled, simple modules. State-dependent and causal versions of our method are also proposed.

We consider using an ensemble of binary classifiers for transductive prediction, when unlabeled test data are known in advance. We derive minimax optimal rules for confidence-rated prediction in this setting. By using PAC-Bayes analysis on these rules, we obtain data-dependent performance guarantees without distributional assumptions on the data. Our analysis techniques are readily extended to a setting in which the predictor is allowed to abstain.

--This article proposes the use of V ector Symbolic Architectures for implementing Hierarchical Graph Neuron, an architecture for memorizing patterns of generic sensor stimuli. The adoption of a V ector Symbolic representation ensures a one-layered design for the approach, while maintaining the previously reported properties and performance characteristics of Hierarchical Graph Neuron, and also improving the noise resistance of the architecture. The proposed architecture enables a linear (with respect to the number of stored entries) time search for an arbitrary sub-pattern. RAPH Neuron (GN) is an approach for memorizing patterns of generic sensor stimuli for later template matching. It is based on the hypothesis that a better associative memory resource can be created by changing the emphasis from high speed sequential CPU processing to parallel network-centric processing [2], [3]. In contrast to contemporary machine learning approaches, GN allows introduction of new patterns in the learning set without the need for retraining. Whilst doing so, it exhibits a high level of scalability i.e. its performance and accuracy do not degrade as the number of stored patterns increases over time. V ector Symbolic Architectures (VSA) [4] are a bio-inspired method of representing concepts and their meaning for modeling cognitive reasoning. It exhibits a set of unique properties which make it suitable for implementation of artificial general intelligence [5], [6], [7], and so, creation of complex systems for sensing and pattern recognition without reliance on complex computation. In the biological world, extremely successful applications of these approaches can be found.

We consider a situation in which we see samples in $\mathbb{R}^d$ drawn i.i.d. from some distribution with mean zero and unknown covariance A. We wish to compute the top eigenvector of A in an incremental fashion - with an algorithm that maintains an estimate of the top eigenvector in O(d) space, and incrementally adjusts the estimate with each new data point that arrives. Two classical such schemes are due to Krasulina (1969) and Oja (1983). We give finite-sample convergence rates for both.

Using nonparametric methods has been increasingly explored in Bayesian hierarchical modeling as a way to increase model flexibility. Although the field shows a lot of promise, inference in many models, including Hierachical Dirichlet Processes (HDP), remain prohibitively slow. One promising path forward is to exploit the submodularity inherent in Indian Buffet Process (IBP) to derive near-optimal solutions in polynomial time. In this work, I will present a brief tutorial on Bayesian nonparametric methods, especially as they are applied to topic modeling. I will show a comparison between different non-parametric models and the current state-of-the-art parametric model, Latent Dirichlet Allocation (LDA).

The problem of inference in a Markov random field (MRF) arises in many applied domains, e.g. in machine learning, computer vision, natural language processing, etc. In this paper we focus on one important type of inference: maximum a posteriori (MAP) inference, often referred to as MRF energy minimization. Inference of this type is a combinatorial optimization problem, i.e. an optimization problem with the finite domain. The most studied case of MRF energy minimization is the situation when the energy can be represented as a sum of terms (potentials) that depend on only one or two variables each (unary and pairwise potentials). In this setting the energy is said to be defined by a graph where the nodes correspond to the variables and the edges to the pairwise potentials. Minimization of energies defined on graphs in known to be NPhard in general [8] but can be done exactly in polynomial time in a number of special cases, e.g. if the graph defining the energy is acyclic [36] or if the energy is submodular in standard [28] or multi-label sense [10]. One way to go beyond pairwise potentials is to add higher-order summands to the energy. For example, Kohli et al. [23] and Ladický et al. [32] use high-order potentials based on superpixels (image regions) for semantic image segmentation; Delong et al. [11] use label cost potentials for geometric model fitting tasks. To be tractable, high-order potentials need to have a compact representation.

In many problems arising in the natural sciences, technology, business and politics, it is crucial to understand the specific connections among the objects under study: for example, the interactions between members of a political party; the firing of synapses in a neuronal network; or citation patterns in reference literature. Mathematically, these objects and their connections are modeled as graphs, and a common goal is to find clusters of similar vertices within a graph. Both model-based and heuristic-based techniques have been proposed for clustering the vertices in a graphs [14, 2, 5, 19]. In this paper we focus on probabilistic performance guarantees for spectral-based techniques which 1 have elements of both model-and heuristic-based methods [18, 20]. We study the consistency of mean squared error clustering via the adjacency spectral embedding for three nested classes of models, each an examples of latent position models [7]: - the stochastic blockmodel where vertices in the same cluster are stochastically equivalent [8], - the degree-corrected stochastic blockmodel where stochastic equivalence holds up to a scaling factor [9], - and the random dot product graph where a natural vertex clustering may not exist [27].

Progress in language and image understanding by machines has sparkled the interest of the research community in more open-ended, holistic tasks, and refueled an old AI dream of building intelligent machines. We discuss a few prominent challenges that characterize such holistic tasks and argue for "question answering about images" as a particular appealing instance of such a holistic task. In particular, we point out that it is a version of a Turing Test that is likely to be more robust to over-interpretations and contrast it with tasks like grounding and generation of descriptions. Finally, we discuss tools to measure progress in this field.

The computation of the global minimum energy conformation (GMEC) is an important and challenging topic in structure-based computational protein design. In this paper, we propose a new protein design algorithm based on the AND/OR branch-and-bound (AOBB) search, which is a variant of the traditional branch-and-bound search algorithm, to solve this combinatorial optimization problem. By integrating with a powerful heuristic function, AOBB is able to fully exploit the graph structure of the underlying residue interaction network of a backbone template to significantly accelerate the design process. Tests on real protein data show that our new protein design algorithm is able to solve many prob- lems that were previously unsolvable by the traditional exact search algorithms, and for the problems that can be solved with traditional provable algorithms, our new method can provide a large speedup by several orders of magnitude while still guaranteeing to find the global minimum energy conformation (GMEC) solution.