Goto

Collaborating Authors

 Education


Evaluating Personality Traits in Large Language Models: Insights from Psychological Questionnaires

arXiv.org Artificial Intelligence

Psychological assessment tools have long helped humans understand Understanding the behaviour of LLMs is essential as they are increasingly behavioural patterns. While Large Language Models (LLMs) used in diverse fields such as education, law, business can generate content comparable to that of humans, we explore and medicine[9] where they significantly influence human interactions whether they exhibit personality traits. To this end, this work applies and decision-making processes. These models can generate psychological tools to LLMs in diverse scenarios to generate coherent and insightful content, allowing personal recommendation personality profiles. Using established trait-based questionnaires and solving complex problems[12]. However, concern for such as the Big Five Inventory and by addressing the possibility of ethical considerations, inherent bias and the potential for misuse training data contamination, we examine the dimensional variability still exist[9] which must be addressed by exploring the underlying and dominance of LLMs across five core personality dimensions: patterns through systematic approaches such as psychological Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism.


A Tutorial On Intersectionality in Fair Rankings

arXiv.org Artificial Intelligence

We address the critical issue of biased algorithms and unfair rankings, which have permeated various sectors, including search engines, recommendation systems, and workforce management. These biases can lead to discriminatory outcomes in a data-driven world, especially against marginalized and underrepresented groups. Efforts towards responsible data science and responsible artificial intelligence aim to mitigate these biases and promote fairness, diversity, and transparency. However, most fairness-aware ranking methods singularly focus on protected attributes such as race, gender, or socio-economic status, neglecting the intersectionality of these attributes, i.e., the interplay between multiple social identities. Understanding intersectionality is crucial to ensure that existing inequalities are not preserved by fair rankings. We offer a description of the main ways to incorporate intersectionality in fair ranking systems through practical examples and provide a comparative overview of existing literature and a synoptic table summarizing the various methodologies. Our analysis highlights the need for intersectionality to attain fairness, while also emphasizing that fairness, alone, does not necessarily imply intersectionality.


Revisiting Online Learning Approach to Inverse Linear Optimization: A Fenchel$-$Young Loss Perspective and Gap-Dependent Regret Analysis

arXiv.org Artificial Intelligence

Linear optimization is arguably the most widely used model of decision-making. Inverse linear optimization is its inverse problem, where the goal is to infer linear objective functions from observed outcomes. Since the early development in geographical science (Tarantola, 1988; Burton and Toint, 1992), inverse linear optimization has been an important subject of study (Ahuja and Orlin, 2001; Heuberger, 2004; Chan et al., 2019) and used in various applications, ranging from route recommendation to healthcare (Chan et al., 2023). Inverse linear optimization is particularly interesting when forward linear optimization is a decision-making model of a human agent.1 Then, the linear objective function represents the agent's internal preference. If the agent repeatedly takes an action upon facing a set of feasible actions, inverse linear optimization can be seen as online learning of the agent's internal preference from pairs of the feasible sets and the agent's choices. Bärmann et al. (2017) studied this setting and proposed an elegant approach based on online learning, which is the focus of this paper and is described below. Consider an agent who addresses decision problems of the following linear-optimization form for = 1,...,: maximize


Robust Conformal Outlier Detection under Contaminated Reference Data

arXiv.org Machine Learning

This paper studies the problem of outlier detection: given a reference dataset (e.g., a collection of legitimate financial transactions) and an unlabeled test point (a new transaction), our goal is to determine whether the test point is an outlier (a fraudulent transaction) by assessing its deviation from the reference data distribution. Naturally, we aim to maximize the detection of outliers by harnessing the capabilities of complex machine learning (ML) models. However, these models typically lack type-I error rate control, potentially resulting in unreliable detections. In our running example, the type-I error is the probability of falsely flagging a legitimate transaction as fraudulent. As such, uncontrolled error rates can lead to costly unnecessary investigations of legitimate transactions and negatively impact customer experience. The broad need for reliable ML systems has sparked a surge of interest in conformal prediction--a versatile framework that can provide statistical guarantees for any "black-box" predictive model [38]. This framework formulates the outlier detection task as a statistical test, where the null hypothesis is that the new data point is not an outlier [23, 5]. To derive a decision rule guaranteeing type-I error control, conformal inference relies on a reference (calibration) set of inlier data points. These points are assumed to be sampled i.i.d.


Exploring the Generalizability of Geomagnetic Navigation: A Deep Reinforcement Learning approach with Policy Distillation

arXiv.org Artificial Intelligence

The advancement in autonomous vehicles has empowered navigation and exploration in unknown environments. Geomagnetic navigation for autonomous vehicles has drawn increasing attention with its independence from GPS or inertial navigation devices. While geomagnetic navigation approaches have been extensively investigated, the generalizability of learned geomagnetic navigation strategies remains unexplored. The performance of a learned strategy can degrade outside of its source domain where the strategy is learned, due to a lack of knowledge about the geomagnetic characteristics in newly entered areas. This paper explores the generalization of learned geomagnetic navigation strategies via deep reinforcement learning (DRL). Particularly, we employ DRL agents to learn multiple teacher models from distributed domains that represent dispersed navigation strategies, and amalgamate the teacher models for generalizability across navigation areas. We design a reward shaping mechanism in training teacher models where we integrate both potential-based and intrinsic-motivated rewards. The designed reward shaping can enhance the exploration efficiency of the DRL agent and improve the representation of the teacher models. Upon the gained teacher models, we employ multi-teacher policy distillation to merge the policies learned by individual teachers, leading to a navigation strategy with generalizability across navigation domains. We conduct numerical simulations, and the results demonstrate an effective transfer of the learned DRL model from a source domain to new navigation areas. Compared to existing evolutionary-based geomagnetic navigation methods, our approach provides superior performance in terms of navigation length, duration, heading deviation, and success rate in cross-domain navigation. Geomagnetic navigation leverages the ubiquitous earth magnetic field signals for the navigation [1], [2], without independence on dedicated devices along the navigation route [3]-[5]. Geomagnetic navigation thus can secure the navigation mission, e.g., in remote areas or underwater environments where there GPS or pre-deployed navigation devices is unavailable [6].


Boosting Multimodal Reasoning with MCTS-Automated Structured Thinking

arXiv.org Artificial Intelligence

Multimodal large language models (MLLMs) exhibit impressive capabilities but still face challenges in complex visual reasoning. While recent efforts attempt to enhance MLLMs' reasoning by incorporating OpenAI o1-like structured thinking through explicit search structures or teacher-guided distillation, they often struggle to balance performance and efficiency. A critical limitation is their heavy reliance on extensive data and search spaces, resulting in low-efficiency implicit insight extraction and data utilization. To address this, we propose AStar, an Automated Structured thinking paradigm for multimodal reasoning via Monte Carlo Tree Search (MCTS). AStar automatically derives high-level cognitive reasoning patterns from limited data using MCTS-powered hierarchical structures. Building on these explicit patterns, we design a unified reasoning framework that seamlessly integrates models' internal reasoning capabilities and external reasoning guidelines, enabling efficient inference with minimal tree iterations. This novel paradigm strikes a compelling balance between performance and efficiency. Extensive experiments demonstrate AStar's effectiveness, achieving superior accuracy (54.0$\%$) on the MathVerse benchmark with a 7B backbone, surpassing GPT-4o (50.2$\%$) while maintaining substantial data and computational efficiency.


Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation

arXiv.org Artificial Intelligence

With the advent of large multimodal language models, science is now at a threshold of an AI-based technological transformation. Recently, a plethora of new AI models and tools has been proposed, promising to empower researchers and academics worldwide to conduct their research more effectively and efficiently. This includes all aspects of the research cycle, especially (1) searching for relevant literature; (2) generating research ideas and conducting experimentation; generating (3) text-based and (4) multimodal content (e.g., scientific figures and diagrams); and (5) AI-based automatic peer review. In this survey, we provide an in-depth overview over these exciting recent developments, which promise to fundamentally alter the scientific research process for good. Our survey covers the five aspects outlined above, indicating relevant datasets, methods and results (including evaluation) as well as limitations and scope for future research. Ethical concerns regarding shortcomings of these tools and potential for misuse (fake science, plagiarism, harms to research integrity) take a particularly prominent place in our discussion. We hope that our survey will not only become a reference guide for newcomers to the field but also a catalyst for new AI-based initiatives in the area of "AI4Science".


Simplifying Formal Proof-Generating Models with ChatGPT and Basic Searching Techniques

arXiv.org Artificial Intelligence

The challenge of formal proof generation has a rich history, but with modern techniques, we may finally be at the stage of making actual progress in real-life mathematical problems. This paper explores the integration of ChatGPT and basic searching techniques to simplify generating formal proofs, with a particular focus on the miniF2F dataset. We demonstrate how combining a large language model like ChatGPT with a formal language such as Lean, which has the added advantage of being verifiable, enhances the efficiency and accessibility of formal proof generation. Despite its simplicity, our best-performing Lean-based model surpasses all known benchmarks with a 31.15% pass rate. We extend our experiments to include other datasets and employ alternative language models, showcasing our models' comparable performance in diverse settings and allowing for a more nuanced analysis of our results. Our findings offer insights into AI-assisted formal proof generation, suggesting a promising direction for future research in formal mathematical proof.


Does Unsupervised Domain Adaptation Improve the Robustness of Amortized Bayesian Inference? A Systematic Evaluation

arXiv.org Machine Learning

Neural networks are fragile when confronted with data that significantly deviates from their training distribution. This is true in particular for simulation-based inference methods, such as neural amortized Bayesian inference (ABI), where models trained on simulated data are deployed on noisy real-world observations. Recent robust approaches employ unsupervised domain adaptation (UDA) to match the embedding spaces of simulated and observed data. However, the lack of comprehensive evaluations across different domain mismatches raises concerns about the reliability in high-stakes applications. We address this gap by systematically testing UDA approaches across a wide range of misspecification scenarios in both a controlled and a high-dimensional benchmark. We demonstrate that aligning summary spaces between domains effectively mitigates the impact of unmodeled phenomena or noise. However, the same alignment mechanism can lead to failures under prior misspecifications - a critical finding with practical consequences. Our results underscore the need for careful consideration of misspecification types when using UDA techniques to increase the robustness of ABI in practice.


Scalable Oversight for Superhuman AI via Recursive Self-Critiquing

arXiv.org Artificial Intelligence

As AI capabilities increasingly surpass human proficiency in complex tasks, current alignment techniques including SFT and RLHF face fundamental challenges in ensuring reliable oversight. These methods rely on direct human assessment and become untenable when AI outputs exceed human cognitive thresholds. In response to this challenge, we explore two hypotheses: (1) critique of critique can be easier than critique itself, extending the widely-accepted observation that verification is easier than generation to the critique domain, as critique itself is a specialized form of generation; (2) this difficulty relationship is recursively held, suggesting that when direct evaluation is infeasible, performing high-order critiques (e.g., critique of critique of critique) offers a more tractable supervision pathway. To examine these hypotheses, we perform Human-Human, Human-AI, and AI-AI experiments across multiple tasks. Our results demonstrate encouraging evidence supporting these hypotheses and suggest that recursive self-critiquing is a promising direction for scalable oversight.