Goto

Collaborating Authors

 Large Language Model


CusEnhancer: A Zero-Shot Scene and Controllability Enhancement Method for Photo Customization via ResInversion

arXiv.org Artificial Intelligence

Recently remarkable progress has been made in synthesizing realistic human photos using text-to-image diffusion models. However, current approaches face degraded scenes, insufficient control, and suboptimal perceptual identity. We introduce CustomEnhancer, a novel framework to augment existing identity customization models. CustomEnhancer is a zero-shot enhancement pipeline that leverages face swapping techniques, pretrained diffusion model, to obtain additional representations in a zeroshot manner for encoding into personalized models. Through our proposed triple-flow fused PerGeneration approach, which identifies and combines two compatible counter-directional latent spaces to manipulate a pivotal space of personalized model, we unify the generation and reconstruction processes, realizing generation from three flows. Our pipeline also enables comprehensive training-free control over the generation process of personalized models, offering precise controlled personalization for them and eliminating the need for controller retraining for per-model. Besides, to address the high time complexity of null-text inversion (NTI), we introduce ResInversion, a novel inversion method that performs noise rectification via a pre-diffusion mechanism, reducing the inversion time by 129 times. Experiments demonstrate that CustomEnhancer reach SOTA results at scene diversity, identity fidelity, training-free controls, while also showing the efficiency of our ResInversion over NTI. The code will be made publicly available upon paper acceptance.


Parallel Thinking, Sequential Answering: Bridging NAR and AR for Efficient Reasoning

arXiv.org Artificial Intelligence

We study reasoning tasks through a framework that integrates auto-regressive (AR) and non-autoregressive (NAR) language models. AR models, which generate text sequentially, excel at producing coherent outputs but often suffer from slow inference, particularly in reasoning-intensive domains such as mathematics and code, where lengthy chains of thought are required. In contrast, NAR models, such as discrete diffusion models, allow parallel generation and offer substantial speedups, though typically at the cost of reduced output quality. To address these limitations, we introduce a new paradigm in which an NAR model efficiently produces intermediate reasoning traces, which subsequently guide an AR model to deliver precise final answers. Experiments demonstrate that our approach yields significant 26% improvements over strong baselines while substantially reducing inference cost.


SLAM-Free Visual Navigation with Hierarchical Vision-Language Perception and Coarse-to-Fine Semantic Topological Planning

arXiv.org Artificial Intelligence

Abstract-- Conventional SLAM pipelines for legged robot navigation are fragile under rapid motion, calibration demands, and sensor drift, while offering limited semantic reasoning for task-driven exploration. T o deal with these issues, we propose a vision-only, SLAM-free navigation framework that replaces dense geometry with semantic reasoning and lightweight topological representations. And a semantic-probabilistic topological map supports coarse-to-fine planning: LLM-based global reasoning for subgoal selection and vision-based local planning for obstacle avoidance. Integrated with reinforcement-learning locomotion controllers, the framework is deployable across diverse legged robot platforms. Experiments in simulation and real-world settings demonstrate consistent improvements in semantic accuracy, planning quality, and navigation success, while ablation studies further showcase the necessity of both hierarchical perception and fine local planning. This work introduces a new paradigm for SLAM-free, vision-language-driven navigation, shifting robotic exploration from geometry-centric mapping to semantics-driven decision making. Autonomous exploration and navigation remain fundamental challenges for mobile robots in open and unstructured environments.


MI-Fuse: Label Fusion for Unsupervised Domain Adaptation with Closed-Source Large-Audio Language Model

arXiv.org Artificial Intelligence

However, SER in real-world deployments often fails under domain mismatch, where source data are unavailable and powerful LALMs are accessible only through an API. We ask: given only unlabeled target-domain audio and an API-only LALM, can a student model be adapted to outperform the LALM in the target domain? To this end, we propose MI-Fuse, a denoised label fusion framework that supplements the LALM with a source-domain trained SER classifier as an auxiliary teacher. The framework draws multiple stochastic predictions from both teachers, weights their mean distributions by mutual-information-based uncertainty, and stabilizes training with an exponential moving average teacher. Experiments across three public emotion datasets and six cross-domain transfers show consistent gains, with the student surpassing the LALM and outperforming the strongest baseline by 3.9%. This approach strengthens emotion-aware speech systems without sharing source data, enabling realistic adaptation.


Building Information Models to Robot-Ready Site Digital Twins (BIM2RDT): An Agentic AI Safety-First Framework

arXiv.org Artificial Intelligence

ABSTRACT The adoption of cyber-physical systems and jobsite intelligence that connects design models, real-time site sensing, and autonomous field operations can dramatically enhance digital management in the Architecture, Engineering, and Construction (AEC) industry. This paper introduces BIM2RDT (Building Information Models to Robot-Ready Site Digital T wins), an agentic artificial intelligence (AI) framework designed to transform static Building Information Modeling (BIM) into dynamic, robot-ready digital twins (DTs) that prioritize safety during construction execution. The framework bridges the gap between pre-existing BIM data and real-time site conditions by integrating three key data streams: geometric and semantic information from BIM models, real-time activity data from IoT sensor networks, and visual-spatial data collected by quadruped robots during site traversal. The methodology introduces Semantic-Gravity ICP (SG-ICP), a novel point cloud registration algorithm that leverages large language model (LLM) reasoning. This creates an intelligent feedback loop where robot-collected data updates the DT, which in turn optimizes paths for subsequent missions. The framework employs YOLOE open-vocabulary object detection and Shi-Tomasi corner detection to identify and track construction elements while using BIM geometry as robust a priori maps. Major findings from experiments demonstrate SG-ICP's superiority over standard ICP, achieving RMSE reductions of 64.3%-88.3% in alignment across varied scenarios with occluded or sparse features, ensuring physically plausible orientations. HA V integration triggers real-time warnings and tasks upon exceeding exposure limits, enhancing compliance with such standards as ISO 5349-1. PRACTICAL APPLICATIONS Construction sites are becoming increasingly complex with the introduction of new technologies such as reality capture equipment and robots, requiring better tools to streamline adoption, avoid tool sprawl, and ensure worker safety. This research introduces a system that combines robots, smart sensors, and building information modeling (BIM) data to create a "digital twin": an up-to-date virtual copy of a construction site's geometries and safety information. The system uses quadruped robots equipped with cameras and sensors to autonomously walk through construction sites, automatically detecting and tracking objects like equipment, materials, and temporary structures. Unlike traditional approaches that start from scratch, this method leverages existing BIM data as a foundation, making the robots more accurate and efficient at understanding their surroundings. Besides geometric site updates, safety information is also presented in the updated digital twin.


Incorporating LLM Embeddings for Variation Across the Human Genome

arXiv.org Artificial Intelligence

In the past few years, foundation models based on large transformer networks such as Google's BERT (Kenton and Toutanova, 2019) and OpenAI's GPT family (Radford, 2018) have been shown to be invaluable aids for scientific discovery in the analysis of genomic data (Cui et al., 2024; Theodoris et al., 2023; Chen and Zou, 2025). More specifically, foundation models targeted for genomic applications typically comprise of those that are trained on enormous databases of experimental data such as scGPT (Cui et al., 2024), which was trained on transcriptomes from 33 million human cells from 441 different studies or the GeneFormer model (Theodoris et al., 2023), which was trained on 29.9 million human single-cell transcriptomes. On the other hand, foundation models based on pre-training on internet-scale databases of natural language texts may offer distinct advantages, such as potentially taking advantage of niche biological relationships which may be widely documented in scientific literature, but not necessarily be represented experimentally in large-scale genomics datasets. For this reason, some recent works have used the embedding outputs of large-language models (LLMs) such as ChatGPT (Radford, 2018) to encode the biological information contained in text-based gene descriptions, such as those in the NCBI database (Schoch et al., 2020). Notably, Chen and Zou (2025) show that these text-based gene descriptors can be input to GPT-3.5 to obtain gene embeddings that act as features/covariates for standard prediction algorithms, denoted GenePT.


Can Federated Learning Safeguard Private Data in LLM Training? Vulnerabilities, Attacks, and Defense Evaluation

arXiv.org Artificial Intelligence

Fine-tuning large language models (LLMs) with local data is a widely adopted approach for organizations seeking to adapt LLMs to their specific domains. Given the shared characteristics in data across different organizations, the idea of collaboratively fine-tuning an LLM using data from multiple sources presents an appealing opportunity. However, organizations are often reluctant to share local data, making centralized fine-tuning impractical. Federated learning (FL), a privacy-preserving framework, enables clients to retain local data while sharing only model parameters for collaborative training, offering a potential solution. While fine-tuning LLMs on centralized datasets risks data leakage through next-token prediction, the iterative aggregation process in FL results in a global model that encapsulates generalized knowledge, which some believe protects client privacy. In this paper, however, we present contradictory findings through extensive experiments. We show that attackers can still extract training data from the global model, even using straightforward generation methods, with leakage increasing as the model size grows. Moreover, we introduce an enhanced attack strategy tailored to FL, which tracks global model updates during training to intensify privacy leakage. To mitigate these risks, we evaluate privacy-preserving techniques in FL, including differential privacy, regularization-constrained updates and adopting LLMs with safety alignment. Our results provide valuable insights and practical guidelines for reducing privacy risks when training LLMs with FL.


Enhancing Molecular Property Prediction with Knowledge from Large Language Models

arXiv.org Artificial Intelligence

Predicting molecular properties is a critical component of drug discovery. Recent advances in deep learning--particularly Graph Neural Networks (GNNs)--have enabled end-to-end learning from molecular structures, reducing reliance on manual feature engineering. However, while GNNs and self-supervised learning approaches have advanced molecular property prediction (MPP), the integration of human prior knowledge remains indispensable, as evidenced by recent methods that leverage large language models (LLMs) for knowledge extraction. Despite their strengths, LLMs are constrained by knowledge gaps and hallucinations, particularly for less-studied molecular properties. In this work, we propose a novel framework that, for the first time, integrates knowledge extracted from LLMs with structural features derived from pre-trained molecular models to enhance MPP . Our approach prompts LLMs to generate both domain-relevant knowledge and executable code for molecular vectorization, producing knowledge-based features that are subsequently fused with structural representations. We employ three state-of-the-art LLMs--GPT -4o, GPT -4.1, and DeepSeek-R1--for knowledge extraction. Extensive experiments demonstrate that our integrated method outperforms existing approaches, confirming that the combination of LLM-derived knowledge and structural information provides a robust and effective solution for MPP .


An LLM-based Agentic Framework for Accessible Network Control

arXiv.org Artificial Intelligence

Traditional approaches to network management have been accessible only to a handful of highly-trained network operators with significant expert knowledge. This creates barriers for lay users to easily manage their networks without resorting to experts. With recent development of powerful large language models (LLMs) for language comprehension, we design a system to make network management accessible to a broader audience of non-experts by allowing users to converse with networks in natural language. To effectively leverage advancements in LLMs, we propose an agentic framework that uses an intermediate representation to streamline configuration across diverse vendor equipment, retrieves the network state from memory in real-time, and provides an interface for external feedback. We also conduct pilot studies to collect real user data of natural language utterances for network control, and present a visualization interface to facilitate dialogue-driven user interaction and enable large-scale data collection for future development. Preliminary experiments validate the effectiveness of our proposed system components with LLM integration on both synthetic and real user utterances. Through our data collection and visualization efforts, we pave the way for more effective use of LLMs and democratize network control for everyday users.


SAMULE: Self-Learning Agents Enhanced by Multi-level Reflection

arXiv.org Artificial Intelligence

Despite the rapid advancements in LLM agents, they still face the challenge of generating meaningful reflections due to inadequate error analysis and a reliance on rare successful trajectories, especially in complex tasks. In this work, we propose SAMULE, a new framework for self-learning agents powered by a retrospective language model that is trained based on Multi-Level Reflection Synthesis. It first synthesizes high-quality reflections across three complementary levels: Single-Trajectory Learning (micro-level) for detailed error correction; Intra-Task Learning (meso-level) to build error taxonomies across multiple trials of the same task, and Inter-Task Learning (macro-level) to extract transferable insights based on same typed errors from diverse task failures. Then we fine-tune a language model serving as the retrospective model to generate reflections during inference. We further extend our framework to interactive settings through a foresight-based reflection mechanism, enabling agents to proactively reflect and adapt during user interactions by comparing predicted and actual responses. Extensive experiments on three challenging benchmarks - TravelPlanner, NATURAL PLAN, and Tau-bench - demonstrate that our approach significantly outperforms reflection-based baselines. Our results highlight the critical role of well-designed reflection synthesis and failure-centric learning in building self-improving LLM agents.