Deep Learning
Global-focal Adaptation with Information Separation for Noise-robust Transfer Fault Diagnosis
Ren, Junyu, Gan, Wensheng, Zhang, Guangyu, Zhong, Wei, Yu, Philip S.
Rotating machinery [1] is critical in industrial applications, where system reliability is essential to avoid financial losses and safety risks. Therefore, timely fault diagnosis is a crucial engineering priority. Deep learning-based fault diagnosis has achieved remarkable success due to its ability to extract features and model complex nonlinear relationships [2, 3]. However, industrial rotating machines operate under diverse conditions, leading to domain shifts that degrade the diagnostic performance of conventional deep learning methods [4]. Among the powerful artificial intelligence (AI) technologies, transfer learning [5] can address these limitations through cross-task knowledge transfer, where domain adaptation has become a widely adopted technique in fault diagnosis, primarily encompassing metric-based approaches, adversarial frameworks, and their hybrid variants [4, 6]. Currently, cross-domain fault diagnosis methods have been extended to encompass a wider range of diverse and practical application scenarios [7]. Given that source domain data are often more abundant in real-world settings, several studies have proposed multi-source transfer fault diagnosis approaches [8, 9]. For closed-set scenarios, various domain adaptation methods have been developed [10]. Since the label categories between source and target domains may not be completely identical, open-set domain adaptation and partial domain adaptation methods have been developed for fault diagnosis [11].
A Storm-Centric 250 m NEXRAD Level-II Dataset for High-Resolution ML Nowcasting
Machine learning-based precipitation nowcasting relies on high-fidelity radar reflectivity sequences to model the short-term evolution of convective storms. However, the development of models capable of predicting extreme weather has been constrained by the coarse resolution (1-2 km) of existing public radar datasets, such as SEVIR, HKO-7, and GridRad-Severe, which smooth the fine-scale structures essential for accurate forecasting. To address this gap, we introduce Storm250-L2, a storm-centric radar dataset derived from NEXRAD Level-II and GridRad-Severe data. We algorithmically crop a fixed, high-resolution (250 m) window around GridRad-Severe storm tracks, preserve the native polar geometry, and provide temporally consistent sequences of both per-tilt sweeps and a pseudo-composite reflectivity product. The dataset comprises thousands of storm events across the continental United States, packaged in HDF5 tensors with rich context metadata and reproducible manifests.
On-Chain Decentralized Learning and Cost-Effective Inference for DeFi Attack Mitigation
Alhaidari, Abdulrahman, Palanisamy, Balaji, Krishnamurthy, Prashant
Billions of dollars are lost every year in DeFi platforms by transactions exploiting business logic or accounting vulnerabilities. Existing defenses focus on static code analysis, public mempool screening, attacker contract detection, or trusted off-chain monitors, none of which prevents exploits submitted through private relays or malicious contracts that execute within the same block. We present the first decentralized, fully on-chain learning framework that: (i) performs gas-prohibitive computation on Layer-2 to reduce cost, (ii) propagates verified model updates to Layer-1, and (iii) enables gas-bounded, low-latency inference inside smart contracts. A novel Proof-of-Improvement (PoIm) protocol governs the training process and verifies each decentralized micro update as a self-verifying training transaction. Updates are accepted by \textit{PoIm} only if they demonstrably improve at least one core metric (e.g., accuracy, F1-score, precision, or recall) on a public benchmark without degrading any of the other core metrics, while adversarial proposals get financially penalized through an adaptable test set for evolving threats. We develop quantization and loop-unrolling techniques that enable inference for logistic regression, SVM, MLPs, CNNs, and gated RNNs (with support for formally verified decision tree inference) within the Ethereum block gas limit, while remaining bit-exact to their off-chain counterparts, formally proven in Z3. We curate 298 unique real-world exploits (2020 - 2025) with 402 exploit transactions across eight EVM chains, collectively responsible for \$3.74 B in losses.
Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalization
Wang, Changsheng, Chen, Xin, Liu, Sijia, Ding, Ke
Adapting pretrained large language models (LLMs) to code domains via supervised fine-tuning (FT) has been commonly used for code generation. However, we identify a previously underappreciated failure mode, the memorization barrier, where strong memorization of downstream code data in the base model could trap optimization and prevent the standard FT from effectively acquiring new, generalizable code knowledge. To overcome this barrier, we propose the information bottleneck (IB)-guided fine-tuning, termed IB-FT, which applies an IB penalty on hidden representations of the code data to compress spurious, memorized features while preserving task-relevant information. Extensive experiments on two code benchmarks (OriGen and Evol-CodeAlpaca-V1) show that IB-FT substantially alleviates the memorization barrier, improves top-1 performance (Pass@$1$), and yields far more stable gains under the stricter multi-sample metric Pass@$k^{(m)}$ (a problem counts as solved only if at least $m$ of $k$ samples pass unit tests) compared with conventional FT.
InfraGPT Smart Infrastructure: An End-to-End VLM-Based Framework for Detecting and Managing Urban Defects
Mohamed, Ibrahim Sheikh, Omaisan, Abdullah Yahya Abdullah
Infrastructure in smart cities is increasingly monitored by networks of closed circuit television (CCTV) cameras. Roads, bridges and tunnels develop cracks, potholes, and fluid leaks that threaten public safety and require timely repair. Manual inspection is costly and hazardous, and existing automatic systems typically address individual defect types or provide unstructured outputs that cannot directly guide maintenance crews. This paper proposes a comprehensive pipeline that leverages street CCTV streams for multi defect detection and segmentation using the YOLO family of object detectors and passes the detections to a vision language model (VLM) for scene aware summarization. The VLM generates a structured action plan in JSON format that includes incident descriptions, recommended tools, dimensions, repair plans, and urgent alerts. We review literature on pothole, crack and leak detection, highlight recent advances in large vision language models such as QwenVL and LLaVA, and describe the design of our early prototype. Experimental evaluation on public datasets and captured CCTV clips demonstrates that the system accurately identifies diverse defects and produces coherent summaries. We conclude by discussing challenges and directions for scaling the system to city wide deployments.
Convolutional Attention in Betting Exchange Markets
Gonรงalves, Rui, Ribeiro, Vitor Miguel, Chertovskih, Roman, Aguiar, Antรณnio Pedro
This study presents the implementation of a short-term forecasting system for price movements in exchange markets, using market depth data and a systematic procedure to enable a fully automated trading system. The case study focuses on the UK to Win Horse Racing market during the pre-live stage on the world's leading betting exchange, Betfair. Innovative convolutional attention mechanisms are introduced and applied to multiple recurrent neural networks and bi-dimensional convolutional recurrent neural network layers. Additionally, a novel padding method for convolutional layers is proposed, specifically designed for multivariate time series processing. These innovations are thoroughly detailed, along with their execution process. The proposed architectures follow a standard supervised learning approach, involving model training and subsequent testing on new data, which requires extensive pre-processing and data analysis. The study also presents a complete end-to-end framework for automated feature engineering and market interactions using the developed models in production. The key finding of this research is that all proposed innovations positively impact the performance metrics of the classification task under examination, thereby advancing the current state-of-the-art in convolutional attention mechanisms and padding methods applied to multivariate time series problems.
Layer-Aware Influence for Online Data Valuation Estimation
Yang, Ziao, Huang, Longbo, Liu, Hongfu
Data-centric learning emphasizes curating high-quality training samples to boost performance rather than designing new architectures. A central problem is to estimate the influence of training sample efficiently. Prior studies largely focus on static influence measured on a converged model, overlooking how data valuation dynamically changes during optimization. This omission neglects the dynamic nature of sample influence during optimization, especially in deep models. To address the computational burden of frequent influence estimation, we develop a layer-aware online estimator that requires only loss-to-output gradients. This design avoids parameter-level and full-network gradients while preserving ranking fidelity. Extensive experiments across LLM pretraining, fine-tuning, and image classification show our method improves accuracy with substantially lower time and memory cost, making dynamic data curation efficient and scalable in practice.
AMStraMGRAM: Adaptive Multi-cutoff Strategy Modification for ANaGRAM
Schwencke, Nilo, Rousselot, Cyriaque, Shilova, Alena, Furtlehner, Cyril
Recent works have shown that natural gradient methods can significantly outperform standard optimizers when training physics-informed neural networks (PINNs). In this paper, we analyze the training dynamics of PINNs optimized with ANaGRAM, a natural-gradient-inspired approach employing singular value decomposition with cutoff regularization. Building on this analysis, we propose a multi-cutoff adaptation strategy that further enhances ANaGRAM's performance. Experiments on benchmark PDEs validate the effectiveness of our method, which allows to reach machine precision on some experiments. To provide theoretical grounding, we develop a framework based on spectral theory that explains the necessity of regularization and extend previous shown connections with Green's functions theory.
Stratos: An End-to-End Distillation Pipeline for Customized LLMs under Distributed Cloud Environments
Dai, Ziming, Zhang, Tuo, Gao, Fei, Cai, Xingyi, Wang, Xiaofei, Zhang, Cheng, Wang, Wenyu, Zang, Chengjie
The growing industrial demand for customized and cost-efficient large language models (LLMs) is fueled by the rise of vertical, domain-specific tasks and the need to optimize performance under constraints such as latency and budget. Knowledge distillation, as an efficient model compression and transfer technique, offers a feasible solution. However, existing distillation frameworks often require manual intervention and struggle to meet such complex user-defined distillation requirements. To bridge this gap, we propose Stratos, an end-to-end LLM distillation pipeline that automates server and model selection, knowledge distillation, and deployment in distributed cloud environments. Given user-defined constraints on model performance and system budget, Stratos automatically selects Pareto-optimal servers, dynamically matches teacher-student pairs, and adapts distillation strategies based on task complexity to optimize cloud hosting. Experiments show that Stratos produces a student model that achieves four times the accuracy of its GPT-4o teacher baseline on a rare, domain-specific Mahjong reasoning task with reverse synthetic data and knowledge injection. Moreover, it achieves reduced latency and cost without compromising accuracy. These results highlight its promise for vertical-domain LLM deployment.
Algorithmic Primitives and Compositional Geometry of Reasoning in Language Models
Lippl, Samuel, McGee, Thomas, Lopez, Kimberly, Pan, Ziwen, Zhang, Pierce, Ziadi, Salma, Eberle, Oliver, Momennejad, Ida
How do latent and inference time computations enable large language models (LLMs) to solve multi-step reasoning? We introduce a framework for tracing and steering algorithmic primitives that underlie model reasoning. Our approach links reasoning traces to internal activation patterns and evaluates algorithmic primitives by injecting them into residual streams and measuring their effect on reasoning steps and task performance. We consider four benchmarks: Traveling Salesperson Problem (TSP), 3SAT, AIME, and graph navigation. We operationalize primitives by clustering neural activations and labeling their matched reasoning traces. We then apply function vector methods to derive primitive vectors as reusable compositional building blocks of reasoning. Primitive vectors can be combined through addition, subtraction, and scalar operations, revealing a geometric logic in activation space. Cross-task and cross-model evaluations (Phi-4, Phi-4-Reasoning, Llama-3-8B) show both shared and task-specific primitives. Notably, comparing Phi-4 with its reasoning-finetuned variant highlights compositional generalization after finetuning: Phi-4-Reasoning exhibits more systematic use of verification and path-generation primitives. Injecting the associated primitive vectors in Phi-4-Base induces behavioral hallmarks associated with Phi-4-Reasoning. Together, these findings demonstrate that reasoning in LLMs may be supported by a compositional geometry of algorithmic primitives, that primitives transfer cross-task and cross-model, and that reasoning finetuning strengthens algorithmic generalization across domains.