Goto

Collaborating Authors

 Lu, Chen


Pre-Trained Large Language Model Based Remaining Useful Life Transfer Prediction of Bearing

arXiv.org Artificial Intelligence

Accurately predicting the remaining useful life (RUL) of rotating machinery, such as bearings, is crucial for equipment reliability and minimizing unexpected failures in industrial systems. Despite recent advancements, data-driven deep learning methods face challenges in practical industrial settings due to inconsistent data distributions between training and testing phases, and limited generalization capabilities for long-term RUL predictions. To address these issues, we propose LM4RUL, a framework for RUL prediction based on pre-trained Large language Model (LLM). LM4RUL leverages the generalization and reasoning capabilities of LLM to transfer predictive knowledge from pre-training, effectively overcoming data inconsistencies and enhancing prediction accuracy. This represents a meaningful advancement in the artificial intelligence field, being among the first efforts to successfully apply LLM to RUL prediction tasks without the need for additional manual instruction, thereby extending the boundaries of AI applications beyond natural language processing and into complex industrial scenarios. The framework includes the local scale perception representation component, which captures fine-grained bearing degradation trends by tokenizing vibration data, and hybrid embedding learning, which selectively freezes and fine-tunes parameters to model complex nonlinear degradation.


LLM-R: A Framework for Domain-Adaptive Maintenance Scheme Generation Combining Hierarchical Agents and RAG

arXiv.org Artificial Intelligence

The increasing use of smart devices has emphasized the critical role of maintenance in production activities. Interactive Electronic Technical Manuals (IETMs) are vital tools that support the maintenance of smart equipment. However, traditional IETMs face challenges such as transitioning from Graphical User Interfaces (GUIs) to natural Language User Interfaces (LUIs) and managing complex logical relationships. Additionally, they must meet the current demands for higher intelligence. This paper proposes a Maintenance Scheme Generation Method based on Large Language Models (LLM-R). The proposed method includes several key innovations: We propose the Low Rank Adaptation-Knowledge Retention (LORA-KR) loss technology to proportionally adjust mixed maintenance data for fine-tuning the LLM. This method prevents knowledge conflicts caused by mixed data, improving the model's adaptability and reasoning ability in specific maintenance domains, Besides, Hierarchical Task-Based Agent and Instruction-level Retrieval-Augmented Generation (RAG) technologies are adopted to optimize the generation steps and mitigate the phenomenon of hallucination caused by the model's Inability to access contextual information. This enhancement improves the model's flexibility and accuracy in handling known or unknown maintenance objects and maintenance scheme scenarios. To validate the proposed method's effectiveness in maintenance tasks, a maintenance scheme dataset was constructed using objects from different fields. The experimental results show that the accuracy of the maintenance schemes generated by the proposed method reached 91.59%, indicating which improvement enhances the intelligence of maintenance schemes and introduces novel technical approaches for equipment maintenance.


An Outline of Prognostics and Health Management Large Model: Concepts, Paradigms, and Challenges

arXiv.org Artificial Intelligence

Prognosis and Health Management (PHM), critical for ensuring task completion by complex systems and preventing unexpected failures, is widely adopted in aerospace, manufacturing, maritime, rail, energy, etc. However, PHM's development is constrained by bottlenecks like generalization, interpretation and verification abilities. Presently, generative artificial intelligence (AI), represented by Large Model, heralds a technological revolution with the potential to fundamentally reshape traditional technological fields and human production methods. Its capabilities, including strong generalization, reasoning, and generative attributes, present opportunities to address PHM's bottlenecks. To this end, based on a systematic analysis of the current challenges and bottlenecks in PHM, as well as the research status and advantages of Large Model, we propose a novel concept and three progressive paradigms of Prognosis and Health Management Large Model (PHM-LM) through the integration of the Large Model with PHM. Subsequently, we provide feasible technical approaches for PHM-LM to bolster PHM's core capabilities within the framework of the three paradigms. Moreover, to address core issues confronting PHM, we discuss a series of technical challenges of PHM-LM throughout the entire process of construction and application. This comprehensive effort offers a holistic PHM-LM technical framework, and provides avenues for new PHM technologies, methodologies, tools, platforms and applications, which also potentially innovates design, research & development, verification and application mode of PHM. And furthermore, a new generation of PHM with AI will also capably be realized, i.e., from custom to generalized, from discriminative to generative, and from theoretical conditions to practical applications.


Confidence Self-Calibration for Multi-Label Class-Incremental Learning

arXiv.org Artificial Intelligence

The partial label challenge in Multi-Label Class-Incremental Learning (MLCIL) arises when only the new classes are labeled during training, while past and future labels remain unavailable. This issue leads to a proliferation of false-positive errors due to erroneously high confidence multi-label predictions, exacerbating catastrophic forgetting within the disjoint label space. In this paper, we aim to refine multi-label confidence calibration in MLCIL and propose a Confidence Self-Calibration (CSC) approach. Firstly, for label relationship calibration, we introduce a class-incremental graph convolutional network that bridges the isolated label spaces by constructing learnable, dynamically extended label relationship graph. Then, for confidence calibration, we present a max-entropy regularization for each multi-label increment, facilitating confidence self-calibration through the penalization of over-confident output distributions. Our approach attains new state-of-the-art results in MLCIL tasks on both MS-COCO and PASCAL VOC datasets, with the calibration of label confidences confirmed through our methodology.


Query lower bounds for log-concave sampling

arXiv.org Machine Learning

Log-concave sampling has witnessed remarkable algorithmic advances in recent years, but the corresponding problem of proving lower bounds for this task has remained elusive, with lower bounds previously known only in dimension one. In this work, we establish the following query lower bounds: (1) sampling from strongly log-concave and log-smooth distributions in dimension $d\ge 2$ requires $\Omega(\log \kappa)$ queries, which is sharp in any constant dimension, and (2) sampling from Gaussians in dimension $d$ (hence also from general log-concave and log-smooth distributions in dimension $d$) requires $\widetilde \Omega(\min(\sqrt\kappa \log d, d))$ queries, which is nearly sharp for the class of Gaussians. Here $\kappa$ denotes the condition number of the target distribution. Our proofs rely upon (1) a multiscale construction inspired by work on the Kakeya conjecture in geometric measure theory, and (2) a novel reduction that demonstrates that block Krylov algorithms are optimal for this problem, as well as connections to lower bound techniques based on Wishart matrices developed in the matrix-vector query literature.


Rejection sampling from shape-constrained distributions in sublinear time

arXiv.org Machine Learning

We consider the task of generating exact samples from a target distribution, known up to normalization, over a finite alphabet. The classical algorithm for this task is rejection sampling, and although it has been used in practice for decades, there is surprisingly little study of its fundamental limitations. In this work, we study the query complexity of rejection sampling in a minimax framework for various classes of discrete distributions. Our results provide new algorithms for sampling whose complexity scales sublinearly with the alphabet size. When applied to adversarial bandits, we show that a slight modification of the Exp3 algorithm reduces the per-iteration complexity from $\mathcal O(K)$ to $\mathcal O(\log^2 K)$, where $K$ is the number of arms.


Contextual Stochastic Block Model: Sharp Thresholds and Contiguity

arXiv.org Machine Learning

In the simplest version of this problem, given access to a graph, one seeks to cluster the vertices into interpretable communities or groups of vertices, which are believed to reflect latent similarities among the nodes. From a theoretical standpoint, this problem has been extensively analyzed under specific generative assumptions on the observed graph; the most popular generative model in this context is the stochastic block model (SBM) [22]. Inspired by intriguing conjectures arising from the statistical physics community [29], community detection under the stochastic block model has been studied extensively. As a consequence, the precise information theoretic limits for recovering the underlying communities have been derived, and optimal algorithms have been identified in this setting (for a survey of these recent breakthroughs, see [1]). In reality, the practitioner often has access to additional information in the form of node covariates, which complements the graph information.