Statistical Learning
Review for NeurIPS paper: Learning Some Popular Gaussian Graphical Models without Condition Number Bounds
This paper studies the well-known problem of structure learning Gaussian Graphical Models. This is simply the problem of learning the zero-non-zero structure of the "precision" matrix (inverse Covariance matrix) of an unknown gaussian distribution from samples. All known efficient algorithms for the problem suffer from a running time and sample complexity dependence on the condition number of the unknown covariance matrix. This paper gives an algorithm, which, for some covariance matrices that satisfy some structural assumptions (walk summabililty, attractive) gives an efficient algorithm that does not depend on the condition number of the unknown covariance (which can be arbitrarily ill-conditioned under the assumptions) . The reviewers (with clarifications in the author feedback) were convinced of both the motivation and non-triviality of the assumptions made and found the algorithmic contributions of this paper important.
Review for NeurIPS paper: Robust, Accurate Stochastic Optimization for Variational Inference
Summary and Contributions: In this paper, the authors study the stochastic optimization algorithm for variational inference. In particular, the authors argue that existing methods stochastic optimization techniques for variational inference are fragile with respect to the hyperparameters of the optimization algorithm. Mainly, authors argue that the standard stopping rule for a stochastic optimization for variational inference is insufficient. The authors view the SGD algorithm with ELBO objective as a Markov chain with a stationary distribution centered around the true variational posterior. The main contribution of this paper are: a) to use iterate averaging to determine the parameter of the variational posterior.
Review for NeurIPS paper: Robust, Accurate Stochastic Optimization for Variational Inference
The reviewers have pointed out a variety of areas where the paper can be improved. I feel that the authors can address these points in modifying their manuscript for the camera ready, especially by inserting a discussion about how their work ties into Dieuleveut, A., Durmus, A. & Bach, F. (2020). I encourage them to also implement the reviewers other concerns.
Review for NeurIPS paper: Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks
Additional Feedback: Line 1: Change to "Natural Gradient Descent..." Line 10, 11: "the function space" should just be "function space" Line 15: it might be worth pointing out here and/or in the intro that a special kind of data preprocessing (the "Forster transform") is required to get this result for K-FAC in general Line 16, 46: "under some assumptions"/"under specific conditions" should perhaps be replaced with "under some approximating assumptions". AFAIK the "gradient independence assumption" doesn't have any rigorous justification and might not even be true in practice. Line 69: "New insights and perspectives on the natural gradient method" also argues that the empirical Fisher is a poor substitute for the "true" one. Line 71: first quotation make is backwards Line 79: delete "firing" here Line 88: "We normalize each sample by" should be "We normalize each sample so that" Line 90: "we overview" should be "we give an overview of" Line 116: Although the use of damping in the context of NTK theory can be explained this way, damping has a larger role in second order optimization in general (where NTK theory doesn't necessarily apply). The way you are describing it though, it sounds like you are saying its use is fully explained by this theory, and I would suggest you change this.
Review for NeurIPS paper: Understanding Approximate Fisher Information for Fast Convergence of Natural Gradient Descent in Wide Neural Networks
This is a compelling paper which covers a lot of ground while keeping the presentation accessible and engaging for the reader. Interestingly, it finds that the K-FAC approximations match the exact NGD trajectory in function space but not weight space. The paper answers quite a lot of questions which are natural to ask, and (having worked a lot in this area) I found the answers interesting and novel. The reviewers seem to have checked it over pretty carefully and didn't spot any problems. The paper is well written, and the authors have clearly paid a lot of attention to the presentation of the ideas.
Reviews: Continuous-time Models for Stochastic Optimization Algorithms
I have read the rebuttal and I believe the authors have satisfactorily addressed my comments on prior work, so I have increased my rating. The SDE approximation method is well-established. Moreover, Minibatch SGD's continuous approximation has been considered by several prior works, e.g. Summary and review comments: The paper is well-written and one of its strengths in generally good comparison with prior work. The main theoretical results are: * SDE approximation for minibatch SGD and SVRG * Well-posedness of the SDEs * Matching convergence bounds using Lyapunov functions * Interpreting time-dependent adjustments as time-change and landscape-stretching.
Reviews: Continuous-time Models for Stochastic Optimization Algorithms
The paper presents an SDE approximation of mini-batch stochastic gradient descent and stochastic variance reduction gradient descent, two widely used methods, and they derive convergence rates. It presents a nice (i.e., not revolutionary, but still of interest to the community) result that fits within this area. Reviewers have a few suggestions for clarifications/improvements.
ESGSenticNet: A Neurosymbolic Knowledge Base for Corporate Sustainability Analysis
Ong, Keane, Mao, Rui, Xing, Frank, Satapathy, Ranjan, Sulaeman, Johan, Cambria, Erik, Mengaldo, Gianmarco
Evaluating corporate sustainability performance is essential to drive sustainable business practices, amid the need for a more sustainable economy. However, this is hindered by the complexity and volume of corporate sustainability data (i.e. sustainability disclosures), not least by the effectiveness of the NLP tools used to analyse them. To this end, we identify three primary challenges - immateriality, complexity, and subjectivity, that exacerbate the difficulty of extracting insights from sustainability disclosures. To address these issues, we introduce ESGSenticNet, a publicly available knowledge base for sustainability analysis. ESGSenticNet is constructed from a neurosymbolic framework that integrates specialised concept parsing, GPT-4o inference, and semi-supervised label propagation, together with a hierarchical taxonomy. This approach culminates in a structured knowledge base of 44k knowledge triplets - ('halve carbon emission', supports, 'emissions control'), for effective sustainability analysis. Experiments indicate that ESGSenticNet, when deployed as a lexical method, more effectively captures relevant and actionable sustainability information from sustainability disclosures compared to state of the art baselines. Besides capturing a high number of unique ESG topic terms, ESGSenticNet outperforms baselines on the ESG relatedness and ESG action orientation of these terms by 26% and 31% respectively. These metrics describe the extent to which topic terms are related to ESG, and depict an action toward ESG. Moreover, when deployed as a lexical method, ESGSenticNet does not require any training, possessing a key advantage in its simplicity for non-technical stakeholders.
A Statistical Learning Approach for Feature-Aware Task-to-Core Allocation in Heterogeneous Platforms
Pivezhandi, Mohammad, Saifullah, Abusayeed, Modekurthy, Prashant
Optimizing task-to-core allocation can substantially reduce power consumption in multi-core platforms without degrading user experience. However, many existing approaches overlook critical factors such as parallelism, compute intensity, and heterogeneous core types. In this paper, we introduce a statistical learning approach for feature selection that identifies the most influential features - such as core type, speed, temperature, and application-level parallelism or memory intensity - for accurate environment modeling and efficient energy optimization. Our experiments, conducted with state-of-the-art Linux governors and thermal modeling techniques, show that correlation-aware task-to-core allocation lowers energy consumption by up to 10% and reduces core temperature by up to 5 degrees Celsius compared to random core selection. Furthermore, our compressed, bootstrapped regression model improves thermal prediction accuracy by 6% while cutting model parameters by 16%, yielding an overall mean square error reduction of 61.6% relative to existing approaches. We provided results based on superscalar Intel Core i7 12th Gen processors with 14 cores, but validated our method across a diverse set of hardware platforms and effectively balanced performance, power, and thermal demands through statistical feature evaluation.
INRet: A General Framework for Accurate Retrieval of INRs for Shapes
Guan, Yushi, Kwan, Daniel, Liang, Ruofan, Panneer, Selvakumar, Jain, Nilesh, Ahuja, Nilesh, Vijaykumar, Nandita
Implicit neural representations (INRs) have become an important method for encoding various data types, such as 3D objects or scenes, images, and videos. They have proven to be particularly effective at representing 3D content, e.g., 3D scene reconstruction from 2D images, novel 3D content creation, as well as the representation, interpolation, and completion of 3D shapes. With the widespread generation of 3D data in an INR format, there is a need to support effective organization and retrieval of INRs saved in a data store. A key aspect of retrieval and clustering of INRs in a data store is the formulation of similarity between INRs that would, for example, enable retrieval of similar INRs using a query INR. In this work, we propose INRet, a method for determining similarity between INRs that represent shapes, thus enabling accurate retrieval of similar shape INRs from an INR data store. INRet flexibly supports different INR architectures such as INRs with octree grids, triplanes, and hash grids, as well as different implicit functions including signed/unsigned distance function and occupancy field. We demonstrate that our method is more general and accurate than the existing INR retrieval method, which only supports simple MLP INRs and requires the same architecture between the query and stored INRs. Furthermore, compared to converting INRs to other representations (e.g., point clouds or multi-view images) for 3D shape retrieval, INRet achieves higher accuracy while avoiding the conversion overhead.