Goto

Collaborating Authors

 utility score



Modeling Contextual Passage Utility for Multihop Question Answering

Jain, Akriti, Garimella, Aparna

arXiv.org Artificial Intelligence

Multihop Question Answering (QA) requires systems to identify and synthesize information from multiple text passages. While most prior retrieval methods assist in identifying relevant passages for QA, further assessing the utility of the passages can help in removing redundant ones, which may otherwise add to noise and inaccuracies in the generated answers. Existing utility prediction approaches model passage utility independently, overlooking a critical aspect of multihop reasoning: the utility of a passage can be context-dependent, influenced by its relation to other passages - whether it provides complementary information or forms a crucial link in conjunction with others. In this paper, we propose a lightweight approach to model contextual passage utility, accounting for inter-passage dependencies. We fine-tune a small transformer-based model to predict passage utility scores for multihop QA. We leverage the reasoning traces from an advanced reasoning model to capture the order in which passages are used to answer a question and obtain synthetic training data. Through comprehensive experiments, we demonstrate that our utility-based scoring of retrieved passages leads to improved reranking and downstream QA performance compared to relevance-based reranking methods.


CoS: Towards Optimal Event Scheduling via Chain-of-Scheduling

Zhao, Yiming, Tang, Jiwei, Di, Shimin, Zheng, Libin, Yu, Jianxing, Yin, Jian

arXiv.org Artificial Intelligence

Recommending event schedules is a key issue in Event-based Social Networks (EBSNs) in order to maintain user activity. An effective recommendation is required to maximize the user's preference, subjecting to both time and geographical constraints. Existing methods face an inherent trade-off among efficiency, effectiveness, and generalization, due to the NP-hard nature of the problem. This paper proposes the Chain-of-Scheduling (CoS) framework, which activates the event scheduling capability of Large Language Models (LLMs) through a guided, efficient scheduling process. CoS enhances LLM by formulating the schedule task into three atomic stages, i.e., exploration, verification and integration. Then we enable the LLMs to generate CoS autonomously via Knowledge Distillation (KD). Experimental results show that CoS achieves near-theoretical optimal effectiveness with high efficiency on three real-world datasets in a interpretable manner. Moreover, it demonstrates strong zero-shot learning ability on out-of-domain data.


Conditional Forecasts and Proper Scoring Rules for Reliable and Accurate Performative Predictions

Boeken, Philip, Zoeter, Onno, Mooij, Joris M.

arXiv.org Machine Learning

Performative predictions are forecasts which influence the outcomes they aim to predict, undermining the existence of correct forecasts and standard methods of elicitation and estimation. We show that conditioning forecasts on covariates that separate them from the outcome renders the target distribution forecast-invariant, guaranteeing well-posedness of the forecasting problem. However, even under this condition, classical proper scoring rules fail to elicit correct forecasts. We prove a general impossibility result and identify two solutions: (i) in decision-theoretic settings, elicitation of correct and incentive-compatible forecasts is possible if forecasts are separating; (ii) scoring with unbiased estimates of the divergence between the forecast and the induced distribution of the target variable yields correct forecasts. Applying these insights to parameter estimation, conditional forecasts and proper scoring rules enable performatively stable estimation of performatively correct parameters, resolving the issues raised by Perdomo et al. (2020). Our results expose fundamental limits of classical forecast evaluation and offer new tools for reliable and accurate forecasting in performative settings.



We thank reviewers for their constructive comments, please see below for our response

Neural Information Processing Systems

We thank reviewers for their constructive comments, please see below for our response. We will make this clear in the revised version. We will include the new results in the revision. Reviewer#2-1-Why SVT suffers from low accuracy. PC's original privacy guarantee might not hold because the sensitivity of the utility score calculated with greedy search We will make the statement more clear in the revision.



Towards Reliable AI in 6G: Detecting Concept Drift in Wireless Network

Tziouvaras, Athanasios, Fortuna, Carolina, Floros, George, Kolomvatsos, Kostas, Sarigiannidis, Panagiotis, Grobelnik, Marko, Bertalanič, Blaž

arXiv.org Artificial Intelligence

--AI-native 6G networks promise unprecedented automation and performance by embedding machine-learning models throughout the radio access and core segments of the network. However, the non-stationary nature of wireless environments due to infrastructure changes, user mobility, and emerging traffic patterns, induces concept drifts that can quickly degrade these model accuracies. Existing methods in general are very domain specific, or struggle with certain type of concept drift. In this paper, we introduce two unsupervised, model-agnostic, batch concept drift detectors. Both methods compute an expected-utility score to decide when concept drift occurred and if model retraining is warranted, without requiring ground-truth labels after deployment. We validate our framework on two real-world wireless use cases in outdoor fingerprinting for localization and for link-anomaly detection, and demonstrate that both methods are outperforming classical detectors such as ADWIN, DDM, CUSUM by 20-40 percentage points. Additionally, they achieve an F1-score of 0.94 and 1.00 in correctly triggering retraining alarm, thus reducing the false alarm rate by up to 20 percentage points compared to the best classical detectors. Cellular networks have undergone significant transformations since their inception, driven by the pursuit of higher performance, broader capabilities, and innovative services.


Generalizable Heuristic Generation Through Large Language Models with Meta-Optimization

Shi, Yiding, Zhou, Jianan, Song, Wen, Bi, Jieyi, Wu, Yaoxin, Zhang, Jie

arXiv.org Artificial Intelligence

Heuristic design with large language models (LLMs) has emerged as a promising approach for tackling combinatorial optimization problems (COPs). However, existing approaches often rely on manually predefined evolutionary computation (EC) optimizers and single-task training schemes, which may constrain the exploration of diverse heuristic algorithms and hinder the generalization of the resulting heuristics. To address these issues, we propose Meta-Optimization of Heuristics (MoH), a novel framework that operates at the optimizer level, discovering effective optimizers through the principle of meta-learning. Specifically, MoH leverages LLMs to iteratively refine a meta-optimizer that autonomously constructs diverse optimizers through (self-)invocation, thereby eliminating the reliance on a predefined EC optimizer. These constructed optimizers subsequently evolve heuristics for downstream tasks, enabling broader heuristic exploration. Moreover, MoH employs a multi-task training scheme to promote its generalization capability. Experiments on classic COPs demonstrate that MoH constructs an effective and interpretable meta-optimizer, achieving state-of-the-art performance across various downstream tasks, particularly in cross-size settings.


Rethinking Predictive Modeling for LLM Routing: When Simple kNN Beats Complex Learned Routers

Li, Yang

arXiv.org Artificial Intelligence

As large language models (LLMs) grow in scale and specialization, routing--selecting the best model for a given input--has become essential for efficient and effective deployment. While recent methods rely on complex learned routing strategies, their dependence on disparate training data and evaluation setups makes comparison and generalization difficult. In this work, we revisit LLM routing through the lens of simplicity. We show that a well-tuned k-Nearest Neighbors (kNN) approach not only matches but often outperforms state-of-the-art learned routers across diverse tasks. To support systematic evaluation, we introduce a suite of standardized routing benchmarks spanning instruction-following, question-answering, and reasoning tasks, as well as the first multi-modal routing dataset involving visual inputs. Our findings reveal that the locality properties of model performance in embedding space enable simple non-parametric methods to achieve strong routing decisions with lower sample complexity than parametric approaches. This challenges the prevailing trend toward sophisticated architectures and highlights the importance of thoroughly evaluating simple baselines before investing in complex solutions. To support reproducibility and further exploration, we will release all benchmarks and code upon publication.