--Graph signal processing has become an essential tool for analyzing data structured on irregular domains. While conventional graph shift operators (GSOs) are effective for certain tasks, they inherently lack flexibility in modeling dependencies between non-adjacent nodes, limiting their ability to represent complex graph structures. T o address this limitation, this paper proposes the unified extended matrix (UEM) framework, which integrates the extended-adjacency matrix and the unified graph representation matrix through parametric design, so as to be able to flexibly adapt to different graph structures and reveal more graph signal information. Theoretical analysis of the UEM is conducted, demonstrating positive semi-definiteness and eigenvalue monotonicity under specific conditions. Then, we propose graph Fourier transform based on UEM (UEM-GFT), which can adaptively tune spectral properties to enhance signal processing performance. Experimental results on synthetic and real-world datasets demonstrate that the UEM-GFT outperforms existing GSO-based methods in anomaly detection tasks, achieving superior performance across varying network topologies. Index T erms --Graph shift operator, unified extended matrix, graph signal processing, graph Fourier transform based on unified extended matrix.

Large language models (LLMs) have been widely adopted in mathematical optimization in scientific scenarios for their extensive knowledge and advanced reasoning capabilities. Existing methods mainly focus on utilizing LLMs to solve optimization problems in a prompt-based manner, which takes observational feedback as additional textual descriptions. However, due to LLM's \textbf{high sensitivity to the prompts} and \textbf{tendency to get lost in lengthy prompts}, these methods struggle to effectively utilize the {observational} feedback from each optimization step, which severely hinders the applications for real-world scenarios. To address these challenges, we propose a conceptually simple and general {bi-level} optimization method, namely \textbf{G}eneral \textbf{S}cientific \textbf{O}ptimizers (GSO). Specifically, GSO first utilizes inner-level simulators as experimental platforms to evaluate the current solution and provide observational feedback. Then, LLMs serve as knowledgeable and versatile scientists, generating new solutions by refining potential errors from the feedback as the outer-level optimization. Finally, simulations together with the expert knowledge in LLMs are jointly updated with bi-level interactions via model editing. Extensive experiments show that GSO consistently outperforms existing state-of-the-art methods using \textit{six} different LLM backbones on \textit{seven} different tasks, demonstrating the effectiveness and a wide range of applications.

In recent years, the rapid advancement of deep learning has led to significant breakthroughs across a wide range of applications, from computer vision to natural language processing, where hyperparameter optimization (HPO) has become increasingly vital in constructing models that achieve optimal performance. As the demand for HPO has been growing, the computational and time costs associated with it have become a significant bottleneck [1]. In this context, Hyperparameter Importance Assessment (HIA) has emerged as a promising solution. By evaluating the importance weights of individual hyperparameters and their combinations within specific models, HIA provides valuable insights into which hyperparameters most significantly impact model performance [2]. With this understanding, deep learning practitioners can focus on optimizing only those hyperparameters that have a more pronounced effect on performance. For less critical hyperparameters, users can reduce the search space during optimization or even fix them at certain values, thereby saving time in the model optimization process [3]. Although there has been considerable exploration of HIA, most existing studies have primarily focused on introducing new HIA methods or determining the importance rankings of hyperparameters for specific models within certain application scenarios. However, there has been limited exploration of how these insights can be strategically applied to enhance the efficiency of the optimization process. To address the challenges in the current research landscape, this paper aims to use Convolutional Neural Networks (CNNs) as the research case to introduce HIA into the deep learning pipeline, demonstrating that the insights gained from HIA can effectively enhance the efficiency of hyper-Second Conference on Parsimony and Learning (CPAL 2025).

Graph Neural Networks (GNNs) have emerged as a notorious alternative to address learning problems dealing with non-Euclidean datasets. However, although most works assume that the graph is perfectly known, the observed topology is prone to errors stemming from observational noise, graph-learning limitations, or adversarial attacks. If ignored, these perturbations may drastically hinder the performance of GNNs. To address this limitation, this work proposes a robust implementation of GNNs that explicitly accounts for the presence of perturbations in the observed topology. For any task involving GNNs, our core idea is to i) solve an optimization problem not only over the learnable parameters of the GNN but also over the true graph, and ii) augment the fitting cost with a term accounting for discrepancies on the graph. Specifically, we consider a convolutional GNN based on graph filters and follow an alternating optimization approach to handle the (non-differentiable and constrained) optimization problem by combining gradient descent and projected proximal updates. The resulting algorithm is not limited to a particular type of graph and is amenable to incorporating prior information about the perturbations. Finally, we assess the performance of the proposed method through several numerical experiments.

Graph convolutional networks (GCNs) can successfully learn the graph signal representation by graph convolution. The graph convolution depends on the graph filter, which contains the topological dependency of data and propagates data features. However, the estimation errors in the propagation matrix (e.g., the adjacency matrix) can have a significant impact on graph filters and GCNs. In this paper, we study the effect of a probabilistic graph error model on the performance of the GCNs. We prove that the adjacency matrix under the error model is bounded by a function of graph size and error probability. We further analytically specify the upper bound of a normalized adjacency matrix with self-loop added. Finally, we illustrate the error bounds by running experiments on a synthetic dataset and study the sensitivity of a simple GCN under this probabilistic error model on accuracy.

How Real World Ontology can help us in the Data Science World of AI Technology? The World Data Ontology could serve as the Single Source/Point of Truth (SSOT/SPOT), Knowledge and Intelligence, Human and Machine. It could be applied as the world model engine for intelligence, learning, inference, decision-making, complex problem-solving and interaction of man-machine superintelligence, innovated as Trans-AI or Meta-AI. Global Data Ontology (GDO) is the prime Single Source/Point of Truth (SSOT/SPOT). The single source of truth, knowledge and intelligence is the world and its data universe, with its causal entities, forces, relationships, principles, mechanisms, laws and regularities.

Graph learning problems are typically approached by focusing on learning the topology of a single graph when signals from all nodes are available. However, many contemporary setups involve multiple related networks and, moreover, it is often the case that only a subset of nodes is observed while the rest remain hidden. Motivated by this, we propose a joint graph learning method that takes into account the presence of hidden (latent) variables. Intuitively, the presence of the hidden nodes renders the inference task ill-posed and challenging to solve, so we overcome this detrimental influence by harnessing the similarity of the estimated graphs. To that end, we assume that the observed signals are drawn from a Gaussian Markov random field with latent variables and we carefully model the graph similarity among hidden (latent) nodes. Then, we exploit the structure resulting from the previous considerations to propose a convex optimization problem that solves the joint graph learning task by providing a regularized maximum likelihood estimator. Finally, we compare the proposed algorithm with different baselines and evaluate its performance over synthetic and real-world graphs.

A powerful framework for studying graphs is to consider them as geometric graphs: nodes are randomly sampled from an underlying metric space, and any pair of nodes is connected if their distance is less than a specified neighborhood radius. Currently, the literature mostly focuses on uniform sampling and constant neighborhood radius. However, real-world graphs are likely to be better represented by a model in which the sampling density and the neighborhood radius can both vary over the latent space. For instance, in a social network communities can be modeled as densely sampled areas, and hubs as nodes with larger neighborhood radius. In this work, we first perform a rigorous mathematical analysis of this (more general) class of models, including derivations of the resulting graph shift operators. The key insight is that graph shift operators should be corrected in order to avoid potential distortions introduced by the non-uniform sampling. Then, we develop methods to estimate the unknown sampling density in a self-supervised fashion. Finally, we present exemplary applications in which the learnt density is used to 1) correct the graph shift operator and improve performance on a variety of tasks, 2) improve pooling, and 3) extract knowledge from networks. Our experimental findings support our theory and provide strong evidence for our model.

The effective representation, precessing, analysis, and visualization of large-scale structured data over graphs are gaining a lot of attention. So far most of the literature has focused on real-valued signals. However, signals are often sparse in the Fourier domain, and more informative and compact representations for them can be obtained using the complex envelope of their spectral components, as opposed to the original real-valued signals. Motivated by this fact, in this work we generalize graph convolutional neural networks (GCN) to the complex domain, deriving the theory that allows to incorporate a complex-valued graph shift operators (GSO) in the definition of graph filters (GF) and process complex-valued graph signals (GS). The theory developed can handle spatio-temporal complex network processes. We prove that complex-valued GCNs are stable with respect to perturbations of the underlying graph support, the bound of the transfer error and the bound of error propagation through multiply layers. Then we apply complex GCN to power grid state forecasting, power grid cyber-attack detection and localization.

Constraint satisfaction problem (CSP) has been actively used for modeling and solving a wide range of complex real-world problems. However, it has been proven that developing efficient methods for solving CSP, especially for large problems, is very difficult and challenging. Existing complete methods for problem-solving are in most cases unsuitable. Therefore, proposing hybrid CSP-based methods for problem-solving has been of increasing interest in the last decades. This paper aims at proposing a novel approach that combines incomplete and complete CSP methods for problem-solving. The proposed approach takes advantage of the group search algorithm (GSO) and the constraint propagation (CP) methods to solve problems related to the remote sensing field. To the best of our knowledge, this paper represents the first study that proposes a hybridization between an improved version of GSO and CP in the resolution of complex constraint-based problems. Experiments have been conducted for the resolution of object recognition problems in satellite images. Results show good performances in terms of convergence and running time of the proposed CSP-based method compared to existing state-of-the-art methods.