Directed Networks
Causal meets Submodular: Subset Selection with Directed Information
We study causal subset selection with Directed Information as the measure of prediction causality. Two typical tasks, causal sensor placement and covariate selection, are correspondingly formulated into cardinality constrained directed information maximizations. To attack the NP-hard problems, we show that the first problem is submodular while not necessarily monotonic. And the second one is "nearly" submodular. To substantiate the idea of approximate submodularity, we introduce a novel quantity, namely submodularity index (SmI), for general set functions. Moreover, we show that based on SmI, greedy algorithm has performance guarantee for the maximization of possibly non-monotonic and non-submodular functions, justifying its usage for a much broader class of problems. We evaluate the theoretical results with several case studies, and also illustrate the application of the subset selection to causal structure learning.
Poisson-Gamma dynamical systems
Aaron Schein, Hanna Wallach, Mingyuan Zhou
Neural Information Processing Systems http://nips.cc/
One-vs-Each Approximation to Softmax for Scalable Estimation of Probabilities
The softmax representation of probabilities for categorical variables plays a prominent role in modern machine learning with numerous applications in areas such as large scale classification, neural language modeling and recommendation systems. However, softmax estimation is very expensive for large scale inference because of the high cost associated with computing the normalizing constant. Here, we introduce an efficient approximation to softmax probabilities which takes the form of a rigorous lower bound on the exact probability. This bound is expressed as a product over pairwise probabilities and it leads to scalable estimation based on stochastic optimization. It allows us to perform doubly stochastic estimation by subsampling both training instances and class labels. We show that the new bound has interesting theoretical properties and we demonstrate its use in classification problems.
Adaptive optimal training of animal behavior
Ji Hyun Bak, Jung Choi, Ilana Witten, Athena Akrami, Jonathan W. Pillow
Neural Information Processing Systems http://nips.cc/
A Minimax Approach to Supervised Learning
Neural Information Processing Systems http://nips.cc/
Stochastic Gradient Geodesic MCMC Methods
Neural Information Processing Systems http://nips.cc/
Bayesian latent structure discovery from multi-neuron recordings
Scott Linderman, Ryan P. Adams, Jonathan W. Pillow
Neural Information Processing Systems http://nips.cc/
A Unified Approach for Learning the Parameters of Sum-Product Networks
Han Zhao, Pascal Poupart, Geoffrey J. Gordon
Neural Information Processing Systems http://nips.cc/
Fast ฮต-free Inference of Simulation Models with Bayesian Conditional Density Estimation
George Papamakarios, Iain Murray
Neural Information Processing Systems http://nips.cc/
Generalized Correspondence-LDA Models (GC-LDA) for Identifying Functional Regions in the Brain
Timothy Rubin, Oluwasanmi O. Koyejo, Michael N. Jones, Tal Yarkoni
This paper presents Generalized Correspondence-LDA (GC-LDA), a generalization of the Correspondence-LDA model that allows for variable spatial representations to be associated with topics, and increased flexibility in terms of the strength of the correspondence between data types induced by the model. We present three variants of GC-LDA, each of which associates topics with a different spatial representation, and apply them to a corpus of neuroimaging data. In the context of this dataset, each topic corresponds to a functional brain region, where the region's spatial extent is captured by a probability distribution over neural activity, and the region's cognitive function is captured by a probability distribution over linguistic terms. We illustrate the qualitative improvements offered by GC-LDA in terms of the types of topics extracted with alternative spatial representations, as well as the model's ability to incorporate a-priori knowledge from the neuroimaging literature. We furthermore demonstrate that the novel features of GC-LDA improve predictions for missing data.