Goto

Collaborating Authors

 Deep Learning


TriP-LLM: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detection

arXiv.org Artificial Intelligence

Time-series anomaly detection plays a central role across a wide range of application domains. With the increasing proliferation of the Internet of Things (IoT) and smart manufacturing, time-series data has dramatically increased in both scale and dimensionality. This growth has exposed the limitations of traditional statistical methods in handling the high heterogeneity and complexity of such data. Inspired by the recent success of large language models (LLMs) in multimodal tasks across language and vision domains, we propose a novel unsupervised anomaly detection framework: A Tri-Branch Patch-wise Large Language Model Framework for Time-Series Anomaly Detection (TriP-LLM). TriP-LLM integrates local and global temporal features through a triple-branch design comprising Patching, Selecting, and Global modules, to encode the input time-series into patch-wise representations, which are then processed by a frozen, pretrained LLM. A lightweight patch-wise decoder reconstructs the input, from which anomaly scores are derived. We evaluate TriP-LLM on several public benchmark datasets using PATE, a recently proposed threshold-free evaluation metric, and conduct all comparisons within a unified open-source framework to ensure fairness. Experimental results show that TriP-LLM consistently outperforms recent state-of-the-art (SOTA) methods across all datasets, demonstrating strong detection capabilities. Furthermore, through extensive ablation studies, we verify the substantial contribution of the LLM to the overall architecture. Compared to LLM-based approaches using Channel Independence (CI) patch processing, TriP-LLM achieves significantly lower memory consumption, making it more suitable for GPU memory-constrained environments. All code and model checkpoints of TriP-LLM are publicly available on https://github.com/YYZStart/TriP-LLM.git


Transferable Parasitic Estimation via Graph Contrastive Learning and Label Rebalancing in AMS Circuits

arXiv.org Artificial Intelligence

Graph representation learning on Analog-Mixed Signal (AMS) circuits is crucial for various downstream tasks, e.g., parasitic estimation. However, the scarcity of design data, the unbalanced distribution of labels, and the inherent diversity of circuit implementations pose significant challenges to learning robust and transferable circuit representations. To address these limitations, we propose CircuitGCL, a novel graph contrastive learning framework that integrates representation scattering and label rebalancing to enhance transferability across heterogeneous circuit graphs. CircuitGCL employs a self-supervised strategy to learn topology-invariant node embeddings through hyperspherical representation scattering, eliminating dependency on large-scale data. Simultaneously, balanced mean squared error (BMSE) and balanced softmax cross-entropy (BSCE) losses are introduced to mitigate label distribution disparities between circuits, enabling robust and transferable parasitic estimation. Evaluated on parasitic capacitance estimation (edge-level task) and ground capacitance classification (node-level task) across TSMC 28nm AMS designs, CircuitGCL outperforms all state-of-the-art (SOTA) methods, with the $R^2$ improvement of $33.64\% \sim 44.20\%$ for edge regression and F1-score gain of $0.9\times \sim 2.1\times$ for node classification. Our code is available at https://github.com/ShenShan123/CircuitGCL.


EFRame: Deeper Reasoning via Exploration-Filter-Replay Reinforcement Learning Framework

arXiv.org Artificial Intelligence

Recent advances in reinforcement learning (RL) have significantly enhanced the reasoning capabilities of large language models (LLMs). Group Relative Policy Optimization (GRPO), a lightweight variant of Proximal Policy Optimization (PPO), improves efficiency but suffers from limited exploration and training instability, limiting its effectiveness on complex reasoning tasks. To address these challenges, we introduce EFRame, an Exploration-Filter-Replay framework that augments GRPO across three dimensions: additional rollouts enable deeper and more targeted exploration, online filtering removes low-quality samples to stabilize gradients and accelerate training, and experience replay amplifies rare yet informative trajectories for stable convergence. This unified framework establishes a principled training cycle that balances exploration, efficiency, and stability. Experiments on diverse reasoning benchmarks demonstrate that EFRame achieves consistent gains, including a 37.9\% relative improvement on Geometry3K over GRPO. EFRame further supports fine-grained sample categorization and precise entropy control, highlighting it as a robust solution for advancing deeper reasoning in LLMs. Our code is available at https://github.com/597358816/EFRame.


Learning to Disentangle Latent Reasoning Rules with Language VAEs: A Systematic Study

arXiv.org Artificial Intelligence

Incorporating explicit reasoning rules within the latent space of language models (LMs) offers a promising pathway to enhance generalisation, interpretability, and controllability. While current Transformer-based language models have shown strong performance on Natural Language Inference (NLI) tasks, they often rely on memorisation rather than rule-based inference. This work investigates how reasoning rules can be explicitly embedded and memorised within the LMs through Language Variational Autoencoders (VAEs). We propose a complete pipeline for learning reasoning rules within Transformer-based language VAEs. This pipeline encompasses three rule-based reasoning tasks, a supporting theoretical framework, and a practical end-to-end architecture. The experiment illustrates the following findings: Disentangled reasoning: Under explicit signal supervision, reasoning rules - viewed as functional mappings - can be disentangled within the encoder's parametric space. This separation results in distinct clustering of rules in the output feature space. Prior knowledge injection: injecting reasoning information into the Query enables the model to more effectively retrieve the stored value Value from memory based on Key. This approach offers a simple method for integrating prior knowledge into decoder-only language models. Performance bottleneck: In mathematical reasoning tasks using Qwen2.5(0.5B), increasing sample count doesn't improve performance beyond a point. Moreover, ffn layers are better than attention layers at preserving the separation of reasoning rules in the model's parameters.


Multi-Scale Finetuning for Encoder-based Time Series Foundation Models

arXiv.org Artificial Intelligence

Time series foundation models (TSFMs) demonstrate impressive zero-shot performance for time series forecasting. However, an important yet underexplored challenge is how to effectively finetune TSFMs on specific downstream tasks. While naive finetuning can yield performance gains, we argue that it falls short of fully leveraging TSFMs' capabilities, often resulting in overfitting and suboptimal performance. Given the diverse temporal patterns across sampling scales and the inherent multi-scale forecasting capabilities of TSFMs, we adopt a causal perspective to analyze finetuning process, through which we highlight the critical importance of explicitly modeling multiple scales and reveal the shortcomings of naive approaches. Focusing on encoder-based TSFMs, we propose Multiscale finetuning (MSFT), a simple yet general framework that explicitly integrates multi-scale modeling into the finetuning process. Experimental results on three different backbones (Moirai, Moment and Units) demonstrate that TSFMs finetuned with MSFT not only outperform naive and typical parameter efficient finetuning methods but also surpass state-of-the-art deep learning methods. Codes are available at https://github.com/zqiao11/MSFT.


Towards Open Foundation Language Model and Corpus for Macedonian: A Low-Resource Language

arXiv.org Artificial Intelligence

The increase in technological adoption worldwide comes with demands for novel tools to be used by the general population. Large Language Models (LLMs) provide a great opportunity in this respect, but their capabilities remain limited for low-resource languages, restricting applications in countries where such languages are spoken. We create several resources to facilitate the adoption of LLMs and to support research advancements for Macedonian. We collect the largest Macedonian corpus to date, consisting of 40GB of textual data and totaling 3.5B words. To support conversational applications, we collect a 106k-instance instruction dataset, carefully built to be culturally grounded. For evaluation, we construct a Macedonian evaluation suite covering seven benchmarks. Finally, we train domestic-yak, a state-of-the-art 8B-parameter model, on our curated datasets and evaluate it against eight baseline models using the newly constructed benchmark suite. Our model outperforms all existing models in the 8B parameter range across all benchmarks, and achieves performance comparable to models up to 10x larger. Furthermore, a qualitative analysis with native speakers reveals that our model is preferred over larger counterparts, receiving higher ratings for grammatical correctness and cultural appropriateness. All datasets, code, and model weights are openly released, setting a foundation for advancing LLMs in similarly underrepresented languages. These resources are publicly available at github.com/LVSTCK for source code, and at huggingface.co/LVSTCK for pretrained model weights and data.


Spatial Deconfounder: Interference-Aware Deconfounding for Spatial Causal Inference

arXiv.org Machine Learning

Causal inference in spatial domains faces two intertwined challenges: (1) unmeasured spatial factors, such as weather, air pollution, or mobility, that confound treatment and outcome, and (2) interference from nearby treatments that violate standard no-interference assumptions. While existing methods typically address one by assuming away the other, we show they are deeply connected: interference reveals structure in the latent confounder. Leveraging this insight, we propose the Spatial Deconfounder, a two-stage method that reconstructs a substitute con-founder from local treatment vectors using a conditional variational autoencoder (CV AE) with a spatial prior, then estimates causal effects via a flexible outcome model. We show that this approach enables nonparametric identification of both direct and spillover effects under weak assumptions--without requiring multiple treatment types or a known model of the latent field. Empirically, we extend SpaCE, a benchmark suite for spatial confounding, to include treatment interference, and show that the Spatial Deconfounder consistently improves effect estimation across real-world datasets in environmental health and social science. By turning interference into a multi-cause signal, our framework bridges spatial and deconfounding literatures to advance robust causal inference in structured data. Causal inference in spatial settings is critical for science and policy, from estimating the health effects of pollution to evaluating land use, climate interventions, and the spread of infectious disease. Most data in these domains are observational, since large-scale interventions are typically infeasible or unethical, so robust methodology is needed to draw valid conclusions. Y et observational studies in these settings face two fundamental challenges that standard methods rarely address together: (1) spillover (interference), where the treatment at one site affects outcomes at nearby sites, violating the Stable Unit Treatment V alue Assumption (SUTV A), and (2) spatially structured unobserved confounding, where latent fields such as weather or socioeconomic context jointly drive exposures and outcomes.


Structured Output Regularization: a framework for few-shot transfer learning

arXiv.org Machine Learning

Transfer learning is often used in deep learning when data is limited, such as in medical imaging applications (Kim et al., 2022). Foundation models, that is large, publicly available, pre-trained models, are often fine-tuned for such tasks where little data is available (Wang et al., 2023; Zhang and Metaxas, 2024; Khan et al., 2025). Beyond freezing part of a model to reduce overfitting, various techniques can increase training data such as data augmentation, and self supervised learning. These methods can reduce overfitting (Chollet, 2021; Wang et al., 2023; Ewen and Khan, 2021), but still struggle when there is little data available (Wang et al., 2023). We propose a new approach, Structured Output Regularization (SOR), a simple framework that adapts and prunes pretrained networks using very little labeled data. Instead of unfreezing internal weights, SOR keeps internal structures frozen, e.g., convolutional filters or higher-level blocks, and regularizes their outputs. Specifically, we freeze internal structure weights, we add new weights between each frozen structure, penalized via lasso penalty to encourage sparsity, and train the network. Structures whose new weights are driven to zero can be removed, yielding a smaller, task-tailored model without training the full parameter set. To regularize the final layer structures, SOR applies group lasso.


'I realised I'd been ChatGPT-ed into bed': how 'Chatfishing' made finding love on dating apps even weirder

The Guardian

'I realised I'd been ChatGPT-ed into bed': how'Chatfishing' made finding love on dating apps even weirder Where once people were duped by soft-focus photos and borrowed chat-up lines, now they have to watch out for computer-generated charm. But it's one thing to use a witty phrase - another thing entirely to build a whole fake persona S tanding outside the pub, 36-year-old business owner Rachel took a final tug on her vape and steeled herself to meet the man she'd spent the last three weeks opening up to. They'd matched on the dating app Hinge and built a rapport that quickly became something deeper. "From the beginning he was asking very open-ended questions, and that felt refreshing," says Rachel. One early message from her match read: "I've been reading a bit about attachment styles lately, it's helped me to understand myself better - and the type of partner I should be looking for. Have you ever looked at yours? Do you know your attachment style?" "It was like he was genuinely trying to get to know me on a deeper level. The questions felt a lot more thoughtful than the usual, 'How's your day going?'"


I asked AI to plan my dream summer holiday. Here's how it turned out

BBC News

I asked AI to plan my dream summer holiday. Here's how it turned out The gothic arches of Santa María de la Asunción crown the hill above the stone harbour where fishermen land their catches of sardines and anchovies. The church in Castro Urdiales, a small seaside town of about 30,000 people in northern Spain, is more than 700 years old. It was the perfect holiday destination for Alan Smith and his family, though he had never heard of it - until he asked ChatGPT. This week a report from the travel association ABTA found an increasing number of people were turning to AI to help with their holidays, from suggesting destinations to planning itineraries once there.