We propose an information-theoretic knowledge distillation approach for the compression of generative adversarial networks, which aims to maximize the mutual information between teacher and student networks via a variational optimization based on an energy-based model. Because the direct computation of the mutual information in continuous domains is intractable, our approach alternatively optimizes the student network by maximizing the variational lower bound of the mutual information. To achieve a tight lower bound, we introduce an energy-based model relying on a deep neural network to represent a flexible variational distribution that deals with high-dimensional images and consider spatial dependencies between pixels, effectively. Since the proposed method is a generic optimization algorithm, it can be conveniently incorporated into arbitrary generative adversarial networks and even dense prediction networks, e.g., image enhancement models. We demonstrate that the proposed algorithm achieves outstanding performance in model compression of generative adversarial networks consistently when combined with several existing models.

We present a novel training approach, named Merge-and-Bound (M&B) for Class Incremental Learning (CIL), which directly manipulates model weights in the parameter space for optimization. Our algorithm involves two types of weight merging: inter-task weight merging and intra-task weight merging. Inter-task weight merging unifies previous models by averaging the weights of models from all previous stages. On the other hand, intra-task weight merging facilitates the learning of current task by combining the model parameters within current stage. For reliable weight merging, we also propose a bounded update technique that aims to optimize the target model with minimal cumulative updates and preserve knowledge from previous tasks; this strategy reveals that it is possible to effectively obtain new models near old ones, reducing catastrophic forgetting. M&B is seamlessly integrated into existing CIL methods without modifying architecture components or revising learning objectives. We extensively evaluate our algorithm on standard CIL benchmarks and demonstrate superior performance compared to state-of-the-art methods.

This paper introduces an active learning framework for manifold Gaussian Process (GP) regression, combining manifold learning with strategic data selection to improve accuracy in high-dimensional spaces. Our method jointly optimizes a neural network for dimensionality reduction and a Gaussian process regressor in the latent space, supervised by an active learning criterion that minimizes global prediction error. Experiments on synthetic data demonstrate superior performance over randomly sequential learning. The framework efficiently handles complex, discontinuous functions while preserving computational tractability, offering practical value for scientific and engineering applications. Future work will focus on scalability and uncertainty-aware manifold learning.

I raised my score from 4 to 6 after reading the author's feedback, mainly due to the novelty of the framework. However, I would expect the author can provide a thorough discussion of the limitation of the result in the camera-ready version. Weakness: Due to the intractbility of the MMD DRO problem, the submission did not find an exact reformulation as much other literature in DRO did for other probability metrics. Instead, the author provides several layers of approximation. The reason why I emphasize the importance of a tight bound, if not an exact reformulation, is that one of the major criticism about (distributionally) robust optimization is that it is sometimes too conservative, and thus a loose upper bound might not be sufficient to mitigate the over-conservativeness and demonstrate the power of distributionally robust optimization. When a new distance is introduced into the DRO framework, a natural question is why it should be used compared with other existing approaches.

We propose an information-theoretic knowledge distillation approach for the compression of generative adversarial networks, which aims to maximize the mutual information between teacher and student networks via a variational optimization based on an energy-based model. Because the direct computation of the mutual information in continuous domains is intractable, our approach alternatively optimizes the student network by maximizing the variational lower bound of the mutual information. To achieve a tight lower bound, we introduce an energy-based model relying on a deep neural network to represent a flexible variational distribution that deals with high-dimensional images and consider spatial dependencies between pixels, effectively. Since the proposed method is a generic optimization algorithm, it can be conveniently incorporated into arbitrary generative adversarial networks and even dense prediction networks, e.g., image enhancement models. We demonstrate that the proposed algorithm achieves outstanding performance in model compression of generative adversarial networks consistently when combined with several existing models.

By leveraging the representations learned in the past, we aim to augment the features Recent deep learning techniques have shown remarkable at each incremental stage to address data deficiency in progress in various computer vision tasks including image the classes belonging to old tasks. To this end, inspired by classification (He et al. 2016; Hu, Shen, and Sun 2018), object adversarial attacks, we adjust the feature representations of detection (Liu et al. 2016; Redmon et al. 2016; Zhu et al. training examples to resemble representations from specific 2021c), semantic segmentation (Chen et al. 2017; Long, target classes that are different from their original classes. Shelhamer, and Darrell 2015; Noh, Hong, and Han 2015), These perturbed features allow a new classifier to maintain and many others. Behind this success is an implicit assumption the decision boundaries for the classes learned up to the that the whole dataset with a predefined set of classes previous stages. Note that this is a novel perspective different should be given in a batch. However, this assumption is from conventional adversarial attack methods (Carlini unlikely to hold in the real-world scenarios which change and Wagner 2017; Goodfellow, Shlens, and Szegedy 2017; dynamically over time. This limits the applicability to realworld Madry et al. 2018; Moosavi-Dezfooli, Fawzi, and Frossard problems because deep neural networks trained under 2016; Zhao, Dua, and Singh 2018), which focus on deceiving changing data distribution often suffer from catastrophic forgetting, models. One may consider generating additional features meaning that the models lose the ability to maintain for each class using the exemplars with the same class labels.

However, they are designed on of connected nodes belong to different classes (Pei et al., the homophilic assumption of graph and clustering 2020; Xie et al., 2023). Traditional GNNs learn representations on heterophilic graph is overlooked. Due to via message passing mechanism under the assumption the lack of labels, it is impossible to first identify of homophily (Fang et al., 2022). Facing heterophilic graphs, a graph as homophilic or heterophilic before a previous approaches mainly suffer two limitations. On the suitable GNN model can be found. Hence, clustering one hand, the local neighbors in a graph are nodes that are on real-world graph with various levels of proximally located, while nodes that are semantically similar homophily poses a new challenge to the graph might be far apart on heterophilic graph (Zhu et al., research community. To fill this gap, we propose 2020). Thus, existing techniques fail to capture long-range a novel graph clustering method, which contains information from distant nodes. On the other hand, they three key components: graph reconstruction, don't distinguish similar and dissimilar neighbors, which a mixed filter, and dual graph clustering carry different amounts of information.

The rise of AI programs like ChatGPT has triggered a tidal wave of ethical handwringing, most prominently from within the industries that it threatens to destroy. After all, just because you can get a robot to instantly write code or write contracts or provide customer support for free, should you? Well, the answer from the Writers Guild of America is a qualified yes. This week, the Writers Guild of America proposed that ChatGPT would absolutely be allowed to write scripts in the future, provided that the credit (and the money) goes to the human writer who came up with the prompts in the first place. The proposal paints a scary picture of the future; a future in which even the most human of arts are crushed under the wheels of an unthinking technology.

The lack of a solid data foundation and solid data workflows is preventing companies from making more progress with machine learning and AI, according to a new Forrester Consulting survey conducted on behalf of Capital One. While companies are having some success in putting machine learning and AI into production, they would be further along if data management issues weren't getting in the way, according to Capital One's new report, "Operationalizing Machine Learning Achieves Key Business Outcomes," which was released today. The report, which is based in part on a July Forrester Consulting survey of 150 data management decision-makers in North America, found that 73% of decision-makers cited transparency, traceability, and explainability of data flows as key issues preventing the operationalizing of machine learning and AI applications. It also found that 57% of those surveyed said internal silos between their data scientists and their practitioners are inhibiting machine learning deployments. "We're still at a point where it's not so much the machine learning algorithm itself that is the roadblock, or the hurdle to folks getting impact," says David Kang, senior vice president and head of data insights at Capital One.

Dr. Guilin Qi is a professor at Southeast University, China, where he also serves as director of the Institute of Cognitive Intelligence and of the Knowledge Science and Engineering Lab. His research interests include knowledge representation and reasoning, knowledge graphs, uncertainty reasoning, and the semantic web. Prof. Qi is an editorial board member of the Journal of Web Semantics, and has co-edited special issues for the Annals of Mathematics and Artificial Intelligence, International Journal of Approximate Reasoning and Journal of Applied Logic. He has over 20 years of research experiences in knowledge engineering and has led many national and industrial projects on knowledge graphs. Prof. Qi has published more than 100 papers on knowledge engineering and knowledge graphs and holds two patents.