Goto

Collaborating Authors

 parenthese





Recommending Composite Items Using Multi-Level Preference Information: A Joint Interaction Modeling Approach

arXiv.org Machine Learning

Recommender systems have become ubiquitous across a wide range of fields, such as ecommerce, media consumption (including movies, books, music, news, etc.), social networks, finance, and many others, due to their effectiveness in identifying relevant items or content among numerous choices [1, 2]. Traditionally, recommender systems, largely based on collaborative filtering techniques, have focused on recommending individual (or "atomic") items, such as movies or books, by understanding users' preferences for these individual items. However, in certain application domains, recommending "composite" items (i.e., combinations of atomic items) represents a very important capability. For illustration, consider a clothing/fashion recommender system, where we want to recommend "outfits" - combinations of tops (t-shirts, shirts, sweaters) and bottoms (pants, skirts, shorts) - to users. In such a case, multiple fashion items in a recommended outfit ideally have to match both functionally and stylistically, which may require domain expertise (e.g., on things like style compatibility) beyond individual preferences. Another key challenge for such recommender systems is that a given user's personal preference for a composite item may not directly translate to the user's personal preferences for the underlying atomic items and vice versa.


Pay Less Attention to Function Words for Free Robustness of Vision-Language Models

arXiv.org Artificial Intelligence

T o address the trade-off between robustness and performance for robust VLM, we observe that function words could incur vulnerability of VLMs against cross-modal adversarial attacks, and propose Function-word De-Attention (FDA) accordingly to mitigate the impact of function words. Similar to differential amplifiers, our FDA calculates the original and the function-word cross-attention within attention heads, and differentially subtracts the latter from the former for more aligned and robust VLMs. Comprehensive experiments include 2 SOTA baselines under 6 different attacks on 2 downstream tasks, 3 datasets, and 3 models. Overall, our FDA yields an average 18/13/53% ASR drop with only 0.2/0.3/0.6% performance drops on the 3 tested models on retrieval, and a 90% ASR drop with a 0.3% performance gain on visual grounding. W e demonstrate the scalability, generalization, and zero-shot performance of FDA experimentally, as well as in-depth ablation studies and analysis. Code will be made publicly available.


Subgroup Validity in Machine Learning for Echocardiogram Data

arXiv.org Artificial Intelligence

Echocardiogram datasets enable training deep learning models to automate interpretation of cardiac ultrasound, thereby expanding access to accurate readings of diagnostically-useful images. However, the gender, sex, race, and ethnicity of the patients in these datasets are underreported and subgroup-specific predictive performance is unevaluated. These reporting deficiencies raise concerns about subgroup validity that must be studied and addressed before model deployment. In this paper, we show that current open echocardiogram datasets are unable to assuage subgroup validity concerns. We improve sociodemographic reporting for two datasets: TMED-2 and MIMIC-IV-ECHO. Analysis of six open datasets reveals no consideration of gender-diverse patients and insufficient patient counts for many racial and ethnic groups. We further perform an exploratory subgroup analysis of two published aortic stenosis detection models on TMED-2. We find insufficient evidence for subgroup validity for sex, racial, and ethnic subgroups. Our findings highlight that more data for underrepresented subgroups, improved demographic reporting, and subgroup-focused analyses are needed to prove subgroup validity in future work.


Mitra: Mixed Synthetic Priors for Enhancing Tabular Foundation Models

arXiv.org Artificial Intelligence

Since the seminal work of TabPFN, research on tabular foundation models (TFMs) based on in-context learning (ICL) has challenged long-standing paradigms in machine learning. Without seeing any real-world data, models pretrained on purely synthetic datasets generalize remarkably well across diverse datasets, often using only a moderate number of in-context examples. This shifts the focus in tabular machine learning from model architecture design to the design of synthetic datasets, or, more precisely, to the prior distributions that generate them. Yet the guiding principles for prior design remain poorly understood. This work marks the first attempt to address the gap. We systematically investigate and identify key properties of synthetic priors that allow pretrained TFMs to generalize well. Based on these insights, we introduce Mitra, a TFM trained on a curated mixture of synthetic priors selected for their diversity, distinctiveness, and performance on real-world tabular data. Mitra consistently outperforms state-of-the-art TFMs, such as TabPFNv2 and TabICL, across both classification and regression benchmarks, with better sample efficiency.


Online SFT for LLM Reasoning: Surprising Effectiveness of Self-Tuning without Rewards

arXiv.org Artificial Intelligence

We present a simple, self-help online supervised finetuning (OSFT) paradigm for LLM reasoning. In this paradigm, the model generates its own responses and is immediately finetuned on this self-generated data. OSFT is a highly efficient training strategy for LLM reasoning, as it is reward-free and uses just one rollout by default. Experiment results show that OSFT achieves downstream performance on challenging mathematical reasoning tasks comparable to strong reinforcement learning with verifiable rewards (RLVR) methods such as GRPO. Our ablation study further demonstrates the efficiency and robustness of OSFT. The major mechanism of OSFT lies in facilitating the model's own existing preference (latent knowledge) learned from pretraining, which leads to reasoning ability improvement. We believe that OSFT offers an efficient and promising alternative to more complex, reward-based training paradigms. Our code is available at https://github.com/ElementQi/OnlineSFT.


Algorithmic Capabilities of Random Transformers

Neural Information Processing Systems

Why is this the case? One possibility is that some aspect of the transformer architecture makes these behaviors easy to learn. Under this hypothesis, transformer models do not implement any useful functionality when initialized; however, their loss landscape is structured such that they can be (computation-and sample-) efficiently optimized for behaviors of interest.


Real-Time Progress Prediction in Reasoning Language Models

arXiv.org Artificial Intelligence

Recent advances in reasoning language models -- particularly those that use long, latent chains of thought -- have demonstrated remarkable capabilities in complex, agentic tasks. However, as these models operate over increasingly extended time horizons, their internal progress becomes opaque to users, complicating expectation management and real-time oversight. In this work, we investigate whether real-time progress prediction is feasible. We discretize progress and train a linear probe to classify reasoning states. We then introduce a two-stage fine-tuning approach that enables reasoning models to generate progress estimates (0$\rightarrow$100\%) during inference. Our best fine-tuned model achieves an average error of 10\% for sequences less than 16,000 tokens, offering a practical mechanism for monitoring and interpreting model reasoning in real time.