We propose an approach to learning the semantics of images which allows usto automatically annotate an image with keywords and to retrieve images based on text queries. We do this using a formalism that models the generation of annotated images. We assume that every image is divided intoregions, each described by a continuous-valued feature vector. Given a training set of images with annotations, we compute a joint probabilistic modelof image features and words which allow us to predict the probability of generating a word given the image regions. This may be used to automatically annotate and retrieve images given a word as a query. Experiments show that our model significantly outperforms the best of the previously reported results on the tasks of automatic image annotation and retrieval.

Classifiers often achieve higher test set accuracy than generative Classifiers (e.g., [6, 4, 14]).

The bootstrap has become a popular method for exploring model (structure) uncertainty. Our experiments with artificial and realworld datademonstrate that the graphs learned from bootstrap samples can be severely biased towards too complex graphical models. Accountingfor this bias is hence essential, e.g., when exploring model uncertainty. We find that this bias is intimately tied to (well-known) spurious dependences induced by the bootstrap. The leading-order bias-correction equals one half of Akaike's penalty for model complexity. We demonstrate the effect of this simple bias-correction in our experiments. We also relate this bias to the bias of the plugin estimator for entropy, as well as to the difference betweenthe expected test and training errors of a graphical model, which asymptotically equals Akaike's penalty (rather than one half).

Most machine learning researchers perform quantitative experiments to estimate generalization error and compare algorithm performances. In order to draw statistically convincing conclusions, it is important to estimate theuncertainty of such estimates. This paper studies the estimation of uncertainty around the K-fold cross-validation estimator. The main theorem shows that there exists no universal unbiased estimator of the variance of K-fold cross-validation. An analysis based on the eigendecomposition ofthe covariance matrix of errors helps to better understand the nature of the problem and shows that naive estimators may grossly underestimate variance, as con£rmed by numerical experiments.

Spectral methods for nonlinear dimensionality reduction (NLDR) impose a neighborhood graph on point data and compute eigenfunctions of a quadratic form generated from the graph.

We propose an information-theoretic clustering approach that incorporates a pre-known partition of the data, aiming to identify common clusters that cut across the given partition. In the standard clustering setting the formation of clusters is guided by a single source of feature information. The newly utilized pre-partition factor introduces an additional bias that counterbalances the impact of the features whenever they become correlated with this known partition. The resulting algorithmic framework was applied successfully to synthetic data, as well as to identifying text-based cross-religion correspondences.

A novel approach to combining clustering and feature selection is presented. Itimplements a wrapper strategy for feature selection, in the sense that the features are directly selected by optimizing the discriminative powerof the used partitioning algorithm. On the technical side, we present an efficient optimization algorithm with guaranteed local convergence property.The only free parameter of this method is selected by a resampling-based stability analysis. Experiments with real-world datasets demonstrate that our method is able to infer both meaningful partitions and meaningful subsets of features.

We present a modified version of the perceptron learning algorithm (PLA) which solves semidefinite programs (SDPs) in polynomial time. The algorithm is based on the following three observations: (i) Semidefinite programs are linear programs with infinitely many (linear) constraints; (ii) every linear program can be solved by a sequence of constraint satisfaction problems with linear constraints; (iii) in general, the perceptron learning algorithm solves a constraint satisfaction problem with linear constraints in finitely many updates. Combining the PLA with a probabilistic rescaling algorithm (which, on average, increases the size of the feasable region) results in a probabilistic algorithmfor solving SDPs that runs in polynomial time. We present preliminary results which demonstrate that the algorithm works,but is not competitive with state-of-the-art interior point methods.

We consider the problem of reconstructing patterns from a feature map. Learning algorithms using kernels to operate in a reproducing kernel Hilbert space (RKHS) express their solutions in terms of input points mapped into the RKHS. We introduce a technique based on kernel principal componentanalysis and regression to reconstruct corresponding patterns inthe input space (aka pre-images) and review its performance in several applications requiring the construction of pre-images. The introduced techniqueavoids difficult and/or unstable numerical optimization, is easy to implement and, unlike previous methods, permits the computation ofpre-images in discrete input spaces.

We present a novel method for approximate inference in Bayesian models andregularized risk functionals. It is based on the propagation of mean and variance derived from the Laplace approximation of conditional probabilitiesin factorizing distributions, much akin to Minka's Expectation Propagation. In the jointly normal case, it coincides with the latter and belief propagation, whereas in the general case, it provides an optimization strategy containing Support Vector chunking, the Bayes Committee Machine, and Gaussian Process chunking as special cases.