Goto

Collaborating Authors

 Statistical Learning


A Preconditioned Interior Point Method for Support Vector Machines Using an ANOVA-Decomposition and NFFT-Based Matrix-Vector Products

arXiv.org Artificial Intelligence

The training of support vector machines (SVMs) leads to large-scale quadratic programs (QPs) [1]. An efficient way to solve these optimization problems is the sequential minimal optimization (SMO) algorithm introduced in [2]. The main motivation for the design of the SMO algorithm comes from the fact that existing optimization methods, i.e., quadratic programming approaches, cannot handle the large-scale dense kernel matrix efficiently. The SMO algorithm is motivated by the result obtained in [3] that showed the optimization problem can be decomposed into the solution of smaller subproblems, which avoids the large-scale dense matrices. When tackling the training as a QP programming task, the use of interior point methods (IPM) has also been studied in the seminal paper of Fine and Scheinberg in [4]: the authors use a low-rank approximation of the kernel matrix and propose a pivoted low-rank Cholesky decomposition to approximate the kernel matrix. A similar matrix approximation was also proposed in [5].


Backbone-based Dynamic Graph Spatio-Temporal Network for Epidemic Forecasting

arXiv.org Artificial Intelligence

Accurate epidemic forecasting is a critical task in controlling disease transmission. Many deep learning-based models focus only on static or dynamic graphs when constructing spatial information, ignoring their relationship. Additionally, these models often rely on recurrent structures, which can lead to error accumulation and computational time consumption. To address the aforementioned problems, we propose a novel model called Backbone-based Dynamic Graph Spatio-Temporal Network (BDGSTN). Intuitively, the continuous and smooth changes in graph structure, make adjacent graph structures share a basic pattern. To capture this property, we use adaptive methods to generate static backbone graphs containing the primary information and temporal models to generate dynamic temporal graphs of epidemic data, fusing them to generate a backbone-based dynamic graph. To overcome potential limitations associated with recurrent structures, we introduce a linear model DLinear to handle temporal dependencies and combine it with dynamic graph convolution for epidemic forecasting. Extensive experiments on two datasets demonstrate that BDGSTN outperforms baseline models and ablation comparison further verifies the effectiveness of model components. Furthermore, we analyze and measure the significance of backbone and temporal graphs by using information metrics from different aspects. Finally, we compare model parameter volume and training time to confirm the superior complexity and efficiency of BDGSTN.


Consensus group decision making under model uncertainty with a view towards environmental policy making

arXiv.org Artificial Intelligence

Group decision making is an important field with interesting applications in various disciplines, among which environmental economics. Group decision, often requires that all or the majority of agents in the group agree to a single proposal or opinion, i.e. consensus. This is particularly true in cases where there is no coercion involved in the implementation of the decision made, so that the implementation of the decision depends on the good will, or rather the acceptance of the common decision by all members of the group. To make the discussion more concrete we consider the following generic situation: Assume that a group of agents, G, has to reach a common decision concerning policies regarding a future contingency X. Policies may refer for instance to the cost of abatement measures for protection against X, which clearly require the acceptance of a commonly acceptable estimate for the value of X by every member of the group as well as the acceptance of a commonly acceptably discount factor. Typically, different member of the group will have different valuations for X, therefore report different costs for the adverse effects of X. Moreover, different members of the group will have different discount rates for calculating the present value of the future adverse effect X.


A framework for mining lifestyle profiles through multi-dimensional and high-order mobility feature clustering

arXiv.org Artificial Intelligence

Human mobility demonstrates a high degree of regularity, which facilitates the discovery of lifestyle profiles. Existing research has yet to fully utilize the regularities embedded in high-order features extracted from human mobility records in such profiling. This study proposes a progressive feature extraction strategy that mines high-order mobility features from users' moving trajectory records from the spatial, temporal, and semantic dimensions. Specific features are extracted such as travel motifs, rhythms decomposed by discrete Fourier transform (DFT) of mobility time series, and vectorized place semantics by word2vec, respectively to the three dimensions, and they are further clustered to reveal the users' lifestyle characteristics. An experiment using a trajectory dataset of over 500k users in Shenzhen, China yields seven user clusters with different lifestyle profiles that can be well interpreted by common sense. The results suggest the possibility of fine-grained user profiling through cross-order trajectory feature engineering and clustering.


Streaming Bayesian Modeling for predicting Fat-Tailed Customer Lifetime Value

arXiv.org Artificial Intelligence

We develop an online learning MCMC approach applicable for hierarchical bayesian models and GLMS. We also develop a fat-tailed LTV model that generalizes over several kinds of fat and thin tails. We demonstrate both developments on commercial LTV data from a large mobile app.


Transfer learning for predicting source terms of principal component transport in chemically reactive flow

arXiv.org Artificial Intelligence

The objective of this study is to evaluate whether the number of requisite training samples can be reduced with the use of various transfer learning models for predicting, for example, the chemical source terms of the data-driven reduced-order model that represents the homogeneous ignition process of a hydrogen/air mixture. Principal component analysis is applied to reduce the dimensionality of the hydrogen/air mixture in composition space. Artificial neural networks (ANNs) are used to tabulate the reaction rates of principal components, and subsequently, a system of ordinary differential equations is solved. As the number of training samples decreases at the target task (i.e.,for T0 > 1000 K and various phi), the reduced-order model fails to predict the ignition evolution of a hydrogen/air mixture. Three transfer learning strategies are then applied to the training of the ANN model with a sparse dataset. The performance of the reduced-order model with a sparse dataset is found to be remarkably enhanced if the training of the ANN model is restricted by a regularization term that controls the degree of knowledge transfer from source to target tasks. To this end, a novel transfer learning method is introduced, parameter control via partial initialization and regularization (PaPIR), whereby the amount of knowledge transferred is systemically adjusted for the initialization and regularization of the ANN model in the target task. It is found that an additional performance gain can be achieved by changing the initialization scheme of the ANN model in the target task when the task similarity between source and target tasks is relatively low.


Quantum Kernel t-Distributed Stochastic Neighbor Embedding

arXiv.org Artificial Intelligence

Data visualization is important in understanding the characteristics of data that are difficult to see directly. It is used to visualize loss landscapes and optimization trajectories to analyze optimization performance. Popular optimization analysis is performed by visualizing a loss landscape around the reached local or global minimum using principal component analysis. However, this visualization depends on the variational parameters of a quantum circuit rather than quantum states, which makes it difficult to understand the mechanism of optimization process through the property of quantum states. Here, we propose a quantum data visualization method using quantum kernels, which enables us to offer fast and highly accurate visualization of quantum states. In our numerical experiments, we visualize hand-written digits dataset and apply $k$-nearest neighbor algorithm to the low-dimensional data to quantitatively evaluate our proposed method compared with a classical kernel method. As a result, our proposed method achieves comparable accuracy to the state-of-the-art classical kernel method, meaning that the proposed visualization method based on quantum machine learning does not degrade the separability of the input higher dimensional data. Furthermore, we visualize the optimization trajectories of finding the ground states of transverse field Ising model and successfully find the trajectory characteristics. Since quantum states are higher dimensional objects that can only be seen via observables, our visualization method, which inherits the similarity of quantum data, would be useful in understanding the behavior of quantum circuits and algorithms.


Duet: efficient and scalable hybriD neUral rElation undersTanding

arXiv.org Artificial Intelligence

Learned cardinality estimation methods have achieved high precision compared to traditional methods. Among learned methods, query-driven approaches have faced the workload drift problem for a long time. Although both data-driven and hybrid methods are proposed to avoid this problem, most of them suffer from high training and estimation costs, limited scalability, instability, and long-tail distribution problems on high-dimensional tables, which seriously affects the practical application of learned cardinality estimators. In this paper, we prove that most of these problems are directly caused by the widely used progressive sampling. We solve this problem by introducing predicate information into the autoregressive model and propose Duet, a stable, efficient, and scalable hybrid method to estimate cardinality directly without sampling or any non-differentiable process, which can not only reduce the inference complexity from $O(n)$ to $O(1)$ compared to Naru and UAE but also achieve higher accuracy on high cardinality and high-dimensional tables. Experimental results show that Duet can achieve all the design goals above and be much more practical. Besides, Duet even has a lower inference cost on CPU than that of most learned methods on GPU.


Contrastive Lift: 3D Object Instance Segmentation by Slow-Fast Contrastive Fusion

arXiv.org Artificial Intelligence

Instance segmentation in 3D is a challenging task due to the lack of large-scale annotated datasets. In this paper, we show that this task can be addressed effectively by leveraging instead 2D pre-trained models for instance segmentation. We propose a novel approach to lift 2D segments to 3D and fuse them by means of a neural field representation, which encourages multi-view consistency across frames. The core of our approach is a slow-fast clustering objective function, which is scalable and well-suited for scenes with a large number of objects. Unlike previous approaches, our method does not require an upper bound on the number of objects or object tracking across frames. To demonstrate the scalability of the slow-fast clustering, we create a new semi-realistic dataset called the Messy Rooms dataset, which features scenes with up to 500 objects per scene. Our approach outperforms the state-of-the-art on challenging scenes from the ScanNet, Hypersim, and Replica datasets, as well as on our newly created Messy Rooms dataset, demonstrating the effectiveness and scalability of our slow-fast clustering method.


Generating high-quality 3DMPCs by adaptive data acquisition and NeREF-based radiometric calibration with UGV plant phenotyping system

arXiv.org Artificial Intelligence

An efficient method for next-best-view (NBV) estimation with high accuracy in the case of the limited camera field of view (FOV) was proposed. Abstract: Fusion of three-dimensional (3D) and multispectral (MS) imaging data has a great potential for high-throughput plant phenotyping of structural and biochemical as well as physiological traits simultaneously, which is important for decision support in agriculture and for crop breeders in selecting the best genotypes. However, lacking of 3D data integrity of various plant canopy structures and low-quality of MS images caused by the complex illumination effects make a great challenge, especially at the proximal imaging scale. Therefore, this study proposed a novel approach for adaptive data acquisition and radiometric calibration to generate high-quality 3D multispectral point clouds (3DMPCs) of plants. An efficient next-best-view (NBV) planning method based on an unmanned ground vehicle (UGV) plant phenotyping system with a multisensor-equipped robotic arm was proposed to achieve adaptive data acquisition. The neural reference field (NeREF) was employed to predict the digital number (DN) values of the hemispherical reference for radiometric calibration. For NBV planning, the average total time for single plant at a joint speed of 1.55 rad/s was about 62.8 s, with an average reduction of 18.0% compared to the unplanned. The integrity of the wholeplant data was improved by an average of 23.6% compared to the fixed viewpoints alone. Compared with the ASD measurements, the average root mean square error (RMSE) of the reflectance spectra obtained from 3DMPCs at different regions of interest was 0.08 with an average decrease of 58.93% compared to the results obtained from the single-frame of MS images without 3D radiometric calibration. The 3Dcalibrated plant 3DMPCs improved the predictive accuracy of partial least squares regression (PLSR) for chlorophyll content, with an average increase of 0.07 in the coefficient of determination (R Our approach introduced a fresh perspective on generating high-quality 3DMPCs of plants under the natural light condition, enabling more precise analysis of plant morphological and physiological parameters. Keywords: adaptive data acquisition; 3DMPC; NBV planning; radiometric calibration; NeREF; chlorophyll content 1. Introduction High-throughput plant phenotyping provides an unprecedented way to systematically evaluate plant development and functionality with the precise quantification of morphological, physiological, biochemical, and performance traits over the whole growth period. It can help on decision support in agriculture, for ecological diversity studies, and for crop breeding in the selection of superior genotypes to improve crop performance, and thus revolutionize the agriculture and breeding strategies to meet the future need of agricultural sustainable development (Freschet et al. 2018; Hu and Schmidhalter 2023).