Directed Networks
Reviews: Poisson-Minibatching for Gibbs Sampling with Convergence Rate Guarantees
Summary: This paper introduces Poisson auxiliary variables to facilitate minibatch sampling. The key insight is with the appropriate Poisson parameterization, the joint distribution (Eq. The authors apply this insight to discrete-state Gibbs sampling (Algorithm 2), Metropolis Hastings (Supplement), and continuous-state Gibbs sampling (Alg 3. and 5). The authors also develop spectral gap lower bounds for all proposed Gibbs sampling methods, which provides a rough guideline for choosing a tuning parameter \lambda and comparing the (asymptotic) per iteration runtime of the methods (Table 1). Finally the authors evaluate the Gibbs methods on synthetic data, showing that their proposed method performs similarly to Gibbs while outperforming alternatives.
Review for NeurIPS paper: Bidirectional Convolutional Poisson Gamma Dynamical Systems
Summary and Contributions: The paper presents a new hierarchical Bayesian model -- convolutional Poisson-Gamma Dynamical Systems (conv-PGDS) -- for generating the observed words in a document corpus. Globally, the model assumes there are K "topic filters", D_1, ... D_K, which are distributions over 3-grams from a finite size vocabulary (size V). Each "topic" (indexed by k) has an appearance probability weight v_k 0 for appearing in a document, and we define transition probability vectors \pi_k Given this global structure, the model generates each document iid. To generate a document j, we use a Gamma dynamical system (with transitions \pi) to obtain a sequence of un-normalized membership "weight embeddings", w_j1 ... w_jT, one for each sentence (indexed by t). Each weight embedding vector w_jt indicates the relative weight of topic k across all words in the sentence t.
Reviews: An Adaptive Empirical Bayesian Method for Sparse Deep Learning
This is a novel combination of existing techniques that appears well-formulated with intriguing experimental results. In particular, this work leverages the strengths stochastic gradient MCMC methods with stochastic approximation to form an adaptive empirical Bayesian approach to learning the parameters and hyperparameters of a Bayesian neural network (BNN). My best understanding is that by optimizing the hyperparameters (rather than sampling them), this new method improves upon existing approaches, speeding up inference without sacrificing quality (especially in the model compression domain). Other areas of BNN literature could be cited, but I think the authors were prudent not to distract the reader from the particular area of focus. This work demonstrates considerable theoretical analysis and is supported by intriguing experimental evidence.
Review for NeurIPS paper: Dynamic Fusion of Eye Movement Data and Verbal Narrations in Knowledge-rich Domains
This paper has a lot of content: Interesting cognitive science question of modelling human decision-making, data fusion of texts and eye movements, modelled with a new dynamic Bayesian nonparametric model, and introduces a new sampler for the model. This paper received a special amount of attention, 5 reviews which were needed because the paper makes several different kinds of contributions. Hence it is not a stereotypical good conference paper having one neat idea and presenting convincing theoretical or empirical support for it. Reviewers discussed the paper intensively, concluding that the paper is likely to be interesting at NeurIPS, and since there is not easy fix to make it more suitable to the format such as dividing it into two papers, it is good enough to be accepted though not among the best papers. Clarity can easily be improved by the authors, and additional details added in both the paper and the supplement.
Reviews: Learning Bayesian Networks with Low Rank Conditional Probability Tables
This paper presents a method for structural learning of a BN given observational data. The work is mainly theoretical, and for the proposal some assumptions are taken. A great effort is also given in presenting and develop theoretically the complexity of the algorithm. One of the key points in the proposed algorithm is the use of Fourier basis vectors (coefficients) and how they are applied in the compressed sensing step. I haven't checked thoroughly all the mathematical part, which is the core of the paper.
Attention-Driven Hierarchical Reinforcement Learning with Particle Filtering for Source Localization in Dynamic Fields
Shi, Yiwei, Yang, Mengyue, Zhang, Qi, Zhang, Weinan, Liu, Cunjia, Liu, Weiru
In many real-world scenarios, such as gas leak detection or environmental pollutant tracking, solving the Inverse Source Localization and Characterization problem involves navigating complex, dynamic fields with sparse and noisy observations. Traditional methods face significant challenges, including partial observability, temporal and spatial dynamics, out-of-distribution generalization, and reward sparsity. To address these issues, we propose a hierarchical framework that integrates Bayesian inference and reinforcement learning. The framework leverages an attention-enhanced particle filtering mechanism for efficient and accurate belief updates, and incorporates two complementary execution strategies: Attention Particle Filtering Planning and Attention Particle Filtering Reinforcement Learning. These approaches optimize exploration and adaptation under uncertainty. Theoretical analysis proves the convergence of the attention-enhanced particle filter, while extensive experiments across diverse scenarios validate the framework's superior accuracy, adaptability, and computational efficiency. Our results highlight the framework's potential for broad applications in dynamic field estimation tasks.
WisdomBot: Tuning Large Language Models with Artificial Intelligence Knowledge
Chen, Jingyuan, Wu, Tao, Ji, Wei, Wu, Fei
Large language models (LLMs) have emerged as powerful tools in natural language processing (NLP), showing a promising future of artificial generated intelligence (AGI). Despite their notable performance in the general domain, LLMs have remained suboptimal in the field of education, owing to the unique challenges presented by this domain, such as the need for more specialized knowledge, the requirement for personalized learning experiences, and the necessity for concise explanations of complex concepts. To address these issues, this paper presents a novel LLM for education named WisdomBot, which combines the power of LLMs with educational theories, enabling their seamless integration into educational contexts. To be specific, we harness self-instructed knowledge concepts and instructions under the guidance of Bloom's Taxonomy as training data. To further enhance the accuracy and professionalism of model's response on factual questions, we introduce two key enhancements during inference, i.e., local knowledge base retrieval augmentation and search engine retrieval augmentation during inference. We substantiate the effectiveness of our approach by applying it to several Chinese LLMs, thereby showcasing that the fine-tuned models can generate more reliable and professional responses.
REX: Causal Discovery based on Machine Learning and Explainability techniques
Renero, Jesus, Ochoa, Idoia, Maestre, Roberto
Causal discovery --the process of identifying cause-and-effect relationships from observational data-- is a pivotal challenge in artificial intelligence (AI) and machine learning. Unveiling causal structures enables robust predictions, facilitates counterfactual reasoning, and enhances decision-making processes in complex systems [1]. Traditional methods for causal discovery often rely on statistical tests for independence and structural equation modeling, which may not scale efficiently with high-dimensional data or effectively capture intricate non-linear relationships [2, 3]. In recent years, machine learning models, particularly deep learning architectures, have achieved remarkable success in predictive tasks. However, these models are typically considered "black boxes" due to their lack of interpretability. This opacity has led to a growing interest in explainable AI (XAI) techniques, with Shapley values emerging as a prominent method for interpreting model predictions [4]. Shapley values, grounded in cooperative game theory, provide a principled approach to attributing the contribution of each feature to the output of a model by quantifying the average marginal contribution of a feature across all possible subsets of features [5]. While Shapley values offer valuable insights into feature importance within a model's predictive framework, the link between feature importance and causal influence is non-trivial.