Goto

Collaborating Authors

 Statistical Learning


Context-Aware Language Models for Forecasting Market Impact from Sequences of Financial News

arXiv.org Artificial Intelligence

Financial news plays a critical role in the information diffusion process in financial markets and is a known driver of stock prices. However, the information in each news article is not necessarily self-contained, often requiring a broader understanding of the historical news coverage for accurate interpretation. Further, identifying and incorporating the most relevant contextual information presents significant challenges. In this work, we explore the value of historical context in the ability of large language models to understand the market impact of financial news. We find that historical context provides a consistent and significant improvement in performance across methods and time horizons. To this end, we propose an efficient and effective contextualization method that uses a large LM to process the main article, while a small LM encodes the historical context into concise summary embeddings that are then aligned with the large model's representation space. We explore the behavior of the model through multiple qualitative and quantitative interpretability tests and reveal insights into the value of contextualization. Finally, we demonstrate that the value of historical context in model predictions has real-world applications, translating to substantial improvements in simulated investment performance.


Zero to Autonomy in Real-Time: Online Adaptation of Dynamics in Unstructured Environments

arXiv.org Artificial Intelligence

Abstract--Autonomous robots must go from zero prior knowledge to safe control within seconds to operate in unstructured environments. Abrupt terrain changes, such as a sudden transition to ice, create dynamics shifts that can destabilize planners unless the model adapts in real-time. We present a method for online adaptation that combines function encoders with recursive least squares, treating the function encoder coefficients as latent states updated from streaming odometry. We evaluate our approach on a V an der Pol system to highlight algorithmic behavior, in a Unity simulator for high-fidelity off-road navigation, and on a Clearpath Jackal robot, including on a challenging terrain at a local ice rink. Across these settings, our method improves model accuracy and downstream planning, reducing collisions compared to static and meta-learning baselines. High-speed ground vehicles require dynamics models that evolve as quickly as the terrain itself. When operating near the limits of controllability, even modest prediction errors in ground terrain interaction can lead to instability, skidding, or rollover. This problem is particularly apparent in off-road navigation: transitions such as pavement to loose gravel can change friction properties within seconds, while mixed terrain features introduce variation in the terrain properties that are difficult to accurately predict. Planning frameworks such as Model Predictive Path Integral Control (MPPI) [27] rely on an accurate model of the system dynamics to predict rollouts and select optimal control actions in real-time.


Empowering Clinical Trial Design through AI: A Randomized Evaluation of PowerGPT

arXiv.org Artificial Intelligence

Sample size calculations for power analysis are critical for clinical research and trial design, yet their complexity and reliance on statistical expertise create barriers for many researchers. We introduce PowerGPT, an AI-powered system integrating large language models (LLMs) with statistical engines to automate test selection and sample size estimation in trial design. In a randomized trial to evaluate its effectiveness, PowerGPT significantly improved task completion rates (99.3% vs. 88.9% for test selection, 99.3% vs. 77.8% for sample size calculation) and accuracy (94.1% vs. 55.4% in sample size estimation, p < 0.001), while reducing average completion time (4.0 vs. 9.3 minutes, p < 0.001). These gains were consistent across various statistical tests and benefited both statisticians and non-statisticians as well as bridging expertise gaps. Already under deployment across multiple institutions, PowerGPT represents a scalable AI-driven approach that enhances accessibility, efficiency, and accuracy in statistical power analysis for clinical research.


Structured Information Loss in Network Embeddings

arXiv.org Artificial Intelligence

Low-dimensional network embeddings (e.g., DeepWalk [1], LINE [2], and many others [3, 4]) are known to be very powerful tools for doing machine learning tasks on graph or network data (for an overview, see [5]). These techniques have shown extremely strong performance in all sorts of tasks including link prediction, community detection, node classification, etc. However, our theoretical understanding of these methods lags behind their empirical success. This is unsurprising: deep-learning algorithms learn very delicate and complex graph properties from real network data and tasks (prediction, classification, etc.); it is difficult to capture these complex properties in a theoretically-sound generative model, let alone determine whether said properties are faithfully reproduced by any given low-dimensional representation. Furthermore, most network embedding methods rely on minimizing a loss function over subsamples of the graph; it is rare to find analytic characterizations of what these methods converge to. Recent work has shown theoretical limits on the representation power of arbitrary low-dimensional embeddings [6, 7, 8]. Davison et al [9] show that these representation limits persist even for graphons, a generative model for graphs that can be thought of as probabilistic generalizations of simple graphs. They prove that inner-product-based embeddings "restrict the class of networks for which an informative embedding can be learned, and networks generated from distinct probabilistic models (graphons) can have embeddings which are asymptotically indistinguishable." 1


Geometric Red-Teaming for Robotic Manipulation

arXiv.org Artificial Intelligence

Standard evaluation protocols in robotic manipulation typically assess policy performance over curated, in-distribution test sets, offering limited insight into how systems fail under plausible variation. We introduce Geometric Red-Teaming (GRT), a red-teaming framework that probes robustness through object-centric geometric perturbations, automatically generating CrashShapes -- structurally valid, user-constrained mesh deformations that trigger catastrophic failures in pre-trained manipulation policies. The method integrates a Jacobian field-based deformation model with a gradient-free, simulator-in-the-loop optimization strategy. Across insertion, articulation, and grasping tasks, GRT consistently discovers deformations that collapse policy performance, revealing brittle failure modes missed by static benchmarks. By combining task-level policy rollouts with constraint-aware shape exploration, we aim to build a general purpose framework for structured, object-centric robustness evaluation in robotic manipulation. We additionally show that fine-tuning on individual CrashShapes, a process we refer to as blue-teaming, improves task success by up to 60 percentage points on those shapes, while preserving performance on the original object, demonstrating the utility of red-teamed geometries for targeted policy refinement. Finally, we validate both red-teaming and blue-teaming results with a real robotic arm, observing that simulated CrashShapes reduce task success from 90% to as low as 22.5%, and that blue-teaming recovers performance to up to 90% on the corresponding real-world geometry -- closely matching simulation outcomes. Videos and code can be found on our project website: https://georedteam.github.io/ .


Linear Dimensionality Reduction for Word Embeddings in Tabular Data Classification

arXiv.org Artificial Intelligence

The Engineers' Salary Prediction Challenge requires classifying salary categories into three classes based on tabular data. The job description is represented as a 300-dimensional word embedding incorporated into the tabular features, drastically increasing dimensionality. Additionally, the limited number of training samples makes classification challenging. Linear dimensionality reduction of word embeddings for tabular data classification remains underexplored. This paper studies Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). We show that PCA, with an appropriate subspace dimension, can outperform raw embeddings. LDA without regularization performs poorly due to covariance estimation errors, but applying shrinkage improves performance significantly, even with only two dimensions. We propose Partitioned-LDA, which splits embeddings into equal-sized blocks and performs LDA separately on each, thereby reducing the size of the covariance matrices. Partitioned-LDA outperforms regular LDA and, combined with shrinkage, achieves top-10 accuracy on the competition public leaderboard. This method effectively enhances word embedding performance in tabular data classification with limited training samples.


MTEB-NL and E5-NL: Embedding Benchmark and Models for Dutch

arXiv.org Artificial Intelligence

Recently, embedding resources, including models, benchmarks, and datasets, have been widely released to support a variety of languages. However, the Dutch language remains underrepresented, typically comprising only a small fraction of the published multilingual resources. To address this gap and encourage the further development of Dutch embeddings, we introduce new resources for their evaluation and generation. First, we introduce the Massive Text Embedding Benchmark for Dutch (MTEB-NL), which includes both existing Dutch datasets and newly created ones, covering a wide range of tasks. Second, we provide a training dataset compiled from available Dutch retrieval datasets, complemented with synthetic data generated by large language models to expand task coverage beyond retrieval. Finally, we release a series of E5-NL models compact yet efficient embedding models that demonstrate strong performance across multiple tasks. We make our resources publicly available through the Hugging Face Hub and the MTEB package.


Digital Voices of Survival: From Social Media Disclosures to Support Provisions for Domestic Violence Victims

arXiv.org Artificial Intelligence

Domestic Violence (DV) is a pervasive public health problem characterized by patterns of coercive and abusive behavior within intimate relationships. With the rise of social media as a key outlet for DV victims to disclose their experiences, online self - di sclosure has emerged as a critical yet underexplored avenue for support - seeking. In addition, existing research lacks a comprehensive and nuanced understanding of DV self - disclosure, support provisions, and their connections. To address these gaps, this study proposes a novel c omputational framework for modeling DV support - seeking behavior alongside community support mechanisms. The framework consists of four key components: self - disclosure detection, post clustering, topic summarization, and support extraction and mapping . We implement and evaluate the framework with data collected from relevant social media communities. Our findings not only advance existing knowledge on DV self - disclosure and online support provisions but also enable victim - centered digital interventions.


Deriving the Scaled-Dot-Function via Maximum Likelihood Estimation and Maximum Entropy Approach

arXiv.org Artificial Intelligence

In this paper, we present a maximum likelihood estimation approach to determine the value vector in transformer models. We model the sequence of value vectors, key vectors, and the query vector as a sequence of Gaussian distributions. The variance in each Gaussian distribution depends on the time step, the corresponding key vector, and the query vector. The mean value in each Gaussian distribution depends on the time step, and the corresponding value vector. This analysis may offer a new explanation of the scaled-dot-product function or softmax function used in transformer architectures [1]. Another explanation, inspired by [4], is based on the maximum entropy approach in natural language processing [5]. In this approach, a query vector and key vectors are used to derive the feature functions for the maximum entropy model.


Domain Adaptive SAR Wake Detection: Leveraging Similarity Filtering and Memory Guidance

arXiv.org Artificial Intelligence

Synthetic Aperture Radar (SAR), with its all-weather and wide-area observation capabilities, serves as a crucial tool for wake detection. However, due to its complex imaging mechanism, wake features in SAR images often appear abstract and noisy, posing challenges for accurate annotation. In contrast, optical images provide more distinct visual cues, but models trained on optical data suffer from performance degradation when applied to SAR images due to domain shift. To address this cross-modal domain adaptation challenge, we propose a Similarity-Guided and Memory-Guided Domain Adaptation (termed SimMemDA) framework for unsupervised domain adaptive ship wake detection via instance-level feature similarity filtering and feature memory guidance. Specifically, to alleviate the visual discrepancy between optical and SAR images, we first utilize WakeGAN to perform style transfer on optical images, generating pseudo-images close to the SAR style. Then, instance-level feature similarity filtering mechanism is designed to identify and prioritize source samples with target-like distributions, minimizing negative transfer. Meanwhile, a Feature-Confidence Memory Bank combined with a K-nearest neighbor confidence-weighted fusion strategy is introduced to dynamically calibrate pseudo-labels in the target domain, improving the reliability and stability of pseudo-labels. Finally, the framework further enhances generalization through region-mixed training, strategically combining source annotations with calibrated target pseudo-labels. Experimental results demonstrate that the proposed SimMemDA method can improve the accuracy and robustness of cross-modal ship wake detection tasks, validating the effectiveness and feasibility of the proposed method.