Statistical Learning
Deep Poisson Factor Modeling
Ricardo Henao, Zhe Gan, James Lu, Lawrence Carin
We propose a new deep architecture for topic modeling, based on Poisson Factor Analysis (PFA) modules. The model is composed of a Poisso n distribution to model observed vectors of counts, as well as a deep hierarchy of hidden binary units. Rather than using logistic functions to characteriz e the probability that a latent binary unit is on, we employ a Bernoulli-Poisson link, which allows PFA modules to be used repeatedly in the deep architecture. We al so describe an approach to build discriminative topic models, by adapting PF A modules. We derive efficient inference via MCMC and stochastic variational met hods, that scale with the number of non-zeros in the data and binary units, yieldin g significant efficiency, relative to models based on logistic links. Experim ents on several corpora demonstrate the advantages of our model when compared to rel ated deep models.
Discriminative Robust Transformation Learning
Jiaji Huang, Qiang Qiu, Guillermo Sapiro, Robert Calderbank
This paper proposes a framework for learning features that are robust to data variation, which is particularly important when only a limited number of training samples are available. The framework makes it possible to tradeoff the discriminative value of learned features against the generalization error of the learning algorithm. Robustness is achieved by encouraging the transform that maps data to features to be a local isometry. This geometric property is shown to improve (K,null)-robustness, thereby providing theoretical justification for reductions in generalization error observed in experiments. The proposed optimization framework is used to train standard learning algorithms such as deep neural networks. Experimental results obtained on benchmark datasets, such as labeled faces in the wild, demonstrate the value of being able to balance discrimination and robustness.