We study optimal estimation for sparse principal component analysis when the number of non-zero elements is small but on the same order as the dimension of the data. We employ approximate message passing (AMP) algorithm and its state evolution to analyze what is the information theoretically minimal mean-squared error and the one achieved by AMP in the limit of large sizes. For a special case of rank one and large enough density of non-zeros Deshpande and Montanari [1] proved that AMP is asymptotically optimal. We show that both for low density and for large rank the problem undergoes a series of phase transitions suggesting existence of a region of parameters where estimation is information theoretically possible, but AMP (and presumably every other polynomial algorithm) fails. The analysis of the large rank limit is particularly instructive.

The stochastic gradient (SG) method can minimize an objective function composed of a large number of differentiable functions, or solve a stochastic optimization problem, to a moderate accuracy. The block coordinate descent/update (BCD) method, on the other hand, handles problems with multiple blocks of variables by updating them one at a time; when the blocks of variables are easier to update individually than together, BCD has a lower per-iteration cost. This paper introduces a method that combines the features of SG and BCD for problems with many components in the objective and with multiple (blocks of) variables. Specifically, a block stochastic gradient (BSG) method is proposed for solving both convex and nonconvex programs. At each iteration, BSG approximates the gradient of the differentiable part of the objective by randomly sampling a small set of data or sampling a few functions from the sum term in the objective, and then, using those samples, it updates all the blocks of variables in either a deterministic or a randomly shuffled order. Its convergence for both convex and nonconvex cases are established in different senses. In the convex case, the proposed method has the same order of convergence rate as the SG method. In the nonconvex case, its convergence is established in terms of the expected violation of a first-order optimality condition. The proposed method was numerically tested on problems including stochastic least squares and logistic regression, which are convex, as well as low-rank tensor recovery and bilinear logistic regression, which are nonconvex.

The paper gives a constructive method, based on greedy algorithms, that provides for the classes of functions with small mixed smoothness the best possible in the sense of order approximation error for the $m$-term approximation with respect to the trigonometric system.

Kernel means are frequently used to represent probability distributions in machine learning problems. In particular, the well known kernel density estimator and the kernel mean embedding both have the form of a kernel mean. Unfortunately, kernel means are faced with scalability issues. A single point evaluation of the kernel density estimator, for example, requires a computation time linear in the training sample size. To address this challenge, we present a method to efficiently construct a sparse approximation of a kernel mean. We do so by first establishing an incoherence-based bound on the approximation error, and then noticing that, for the case of radial kernels, the bound can be minimized by solving the $k$-center problem. The outcome is a linear time construction of a sparse kernel mean, which also lends itself naturally to an automatic sparsity selection scheme. We show the computational gains of our method by looking at three problems involving kernel means: Euclidean embedding of distributions, class proportion estimation, and clustering using the mean-shift algorithm.

Past research has challenged us with the task of showing relational patterns between text-based data and then clustering for predictive analysis using Golay Code technique. We focus on a novel approach to extract metaknowledge in multimedia datasets. Our collaboration has been an on-going task of studying the relational patterns between datapoints based on metafeatures extracted from metaknowledge in multimedia datasets. Those selected are significant to suit the mining technique we applied, Golay Code algorithm. In this research paper we summarize findings in optimization of metaknowledge representation for 23-bit representation of structured and unstructured multimedia data in order to

In the presented research project, a software and hardware infrastructure for parking space focussed inter-modal route planning in a public pilot region in Berlin is developed. One central topic is the development of a prediction system which gives an estimated occupancy for the parking spaces in the pilot region for a given date and time in the future. Occupancy data will be collected online by roadside parking sensors developed within the project. The occupancy prediction will be implemented using “Neural Gas” machine learning in combination with a proposed method which uses data threads to improve the prediction quality. In this paper, a short overview of the whole research project is given. Furthermore, the concept of the software framework and the learning methods are presented and first collected data is shown. The prediction method using data threads is explained in more detail.

Among the languages used for representing goals, actions and their consequences on the world for decision making and planning, GDL (Game Description Language) has the ability to represent complex actions in potentially uncertain and competitive environments. The aim of this paper is to exploit stochastic constraint networks in order to provide compact representations of strategic games, and to identify optimal policies in those games with generic forward checking method. From this perspective, we develop a compiler allowing to translate games, described in GDL, into instances of the Stochastic Constraint Optimization Problem (SCSP). Our compiler is proved correct for the class GDL of games with complete information and oblivious environment. The interest of our approach is illustrated by solving several GDL games with a SCSP solver.

We propose a novel online learning method for minimizing regret in large extensive-form games. The approach learns a function approximator online to estimate the regret for choosing a particular action. A no-regret algorithm uses these estimates in place of the true regrets to define a sequence of policies. We prove the approach sound by providing a bound relating the quality of the function approximation and regret of the algorithm. A corollary being that the method is guaranteed to converge to a Nash equilibrium in self-play so long as the regrets are ultimately realizable by the function approximator. Our technique can be understood as a principled generalization of existing work on abstraction in large games; in our work, both the abstraction as well as the equilibrium are learned during self-play. We demonstrate empirically the method achieves higher quality strategies than state-of-the-art abstraction techniques given the same resources.

Recent increase in interest in using rotorcraft (helicopters and tilt-rotor craft) for public transportation has spurred research in making rotorcraft less noisy, particularly as they land. The ground noise associated with landing trajectories followed by rotorcraft depends in part on the changes in altitude and velocity of the rotorcraft during flight. Acoustic models of ground noise taking altitude and velocity effects into account can be used in an optimization process to determine a set of potentially quieter pilot operations. However, optimizing solely for acoustic properties produces patterns that abstract away from the environment in which the trajectory is flown. A quiet procedure flown over a residential area can create considerable annoyance. To overcome this limitation of acoustic-based optimization we propose a hybrid cost model for optimization that combines acoustic criteria with a land use model that views noise-sensitive areas around landing facilities as weighted obstacles. The result is a 3D route planning problem with obstacles. We introduce a system, called NORA (Noise Optimization for Rotorcraft Approach) that allows for the computation of trajectories that simultaneously solve for acoustically quiet patterns that also avoid land sensitive areas.

The presented framework is the The multirobot coalition formation problem seeks to intelligently first to leverage a conceptual clustering technique to partition partition a team of heterogeneous robots into any set of coalition formation algorithms in order to derive coalitions for a set of real-world tasks. Besides being N Pan optimal hierarchy classification tree, given any classification complete (Sandholm et al. 1999), the problem is also hard taxonomy. The results contribute to the state-ofthe-art to approximate (Service and Adams 2011a). Traditional approaches in multiagent systems by demonstrating the existence to solving the problem include a number of greedy of crucial patterns and intricate relationships among existing algorithms (Shehory and Kraus 1998; Vig and Adams coalition algorithms.