Goto

Collaborating Authors

 South America


Vision Language Models in Medicine

arXiv.org Artificial Intelligence

With the advent of Vision-Language Models (VLMs), medical artificial intelligence (AI) has experienced significant technological progress and paradigm shifts. This survey provides an extensive review of recent advancements in Medical Vision-Language Models (Med-VLMs), which integrate visual and textual data to enhance healthcare outcomes. We discuss the foundational technology behind Med-VLMs, illustrating how general models are adapted for complex medical tasks, and examine their applications in healthcare. The transformative impact of Med-VLMs on clinical practice, education, and patient care is highlighted, alongside challenges such as data scarcity, narrow task generalization, interpretability issues, and ethical concerns like fairness, accountability, and privacy. These limitations are exacerbated by uneven dataset distribution, computational demands, and regulatory hurdles. Rigorous evaluation methods and robust regulatory frameworks are essential for safe integration into healthcare workflows. Future directions include leveraging large-scale, diverse datasets, improving cross-modal generalization, and enhancing interpretability. Innovations like federated learning, lightweight architectures, and Electronic Health Record (EHR) integration are explored as pathways to democratize access and improve clinical relevance. This review aims to provide a comprehensive understanding of Med-VLMs' strengths and limitations, fostering their ethical and balanced adoption in healthcare.


Do Emotions Really Affect Argument Convincingness? A Dynamic Approach with LLM-based Manipulation Checks

arXiv.org Artificial Intelligence

Emotions have been shown to play a role in argument convincingness, yet this aspect is underexplored in the natural language processing (NLP) community. Unlike prior studies that use static analyses, focus on a single text domain or language, or treat emotion as just one of many factors, we introduce a dynamic framework inspired by manipulation checks commonly used in psychology and social science; leveraging LLM-based manipulation checks, this framework examines the extent to which perceived emotional intensity influences perceived convincingness. Through human evaluation of arguments across different languages, text domains, and topics, we find that in over half of cases, judgments of convincingness remain unchanged despite variations in perceived emotional intensity; when emotions do have an impact, they more often enhance rather than weaken convincingness. We further analyze how 11 LLMs behave in the same scenario, finding that while LLMs generally mirror human patterns, they struggle to capture nuanced emotional effects in individual judgments.


ARACNE: An LLM-Based Autonomous Shell Pentesting Agent

arXiv.org Artificial Intelligence

The complete automation of cyber-attacks is an area of growing interest since the surge of Large Language Models (LLMs) in recent years. Although the application of LLM in all areas of cybersecurity has flourished, the creation of attacking LLM agents that can act independently is among the most popular options [1]. Attacking LLM agents can perform automatic security testing of applications, lowering the cost for organizations to find vulnerabilities and misconfiguration problems and identify other security issues [2]. Existing automated attacking agents, such as PenHeal [2], AutoAttacker [3], and HackSynth [4] show promising results but with clear limitations. Agents are unable to work so far without occasional mistakes and hallucinations.


Reinforcement Learning-based Approach for Vehicle-to-Building Charging with Heterogeneous Agents and Long Term Rewards

arXiv.org Artificial Intelligence

Strategic aggregation of electric vehicle batteries as energy reservoirs can optimize power grid demand, benefiting smart and connected communities, especially large office buildings that offer workplace charging. This involves optimizing charging and discharging to reduce peak energy costs and net peak demand, monitored over extended periods (e.g., a month), which involves making sequential decisions under uncertainty and delayed and sparse rewards, a continuous action space, and the complexity of ensuring generalization across diverse conditions. Existing algorithmic approaches, e.g., heuristic-based strategies, fall short in addressing real-time decision-making under dynamic conditions, and traditional reinforcement learning (RL) models struggle with large state-action spaces, multi-agent settings, and the need for long-term reward optimization. To address these challenges, we introduce a novel RL framework that combines the Deep Deterministic Policy Gradient approach (DDPG) with action masking and efficient MILP-driven policy guidance. Our approach balances the exploration of continuous action spaces to meet user charging demands. Using real-world data from a major electric vehicle manufacturer, we show that our approach comprehensively outperforms many well-established baselines and several scalable heuristic approaches, achieving significant cost savings while meeting all charging requirements. Our results show that the proposed approach is one of the first scalable and general approaches to solving the V2B energy management challenge.


DocPuzzle: A Process-Aware Benchmark for Evaluating Realistic Long-Context Reasoning Capabilities

arXiv.org Artificial Intelligence

We present DocPuzzle, a rigorously constructed benchmark for evaluating long-context reasoning capabilities in large language models (LLMs). This benchmark comprises 100 expert-level QA problems requiring multi-step reasoning over long real-world documents. To ensure the task quality and complexity, we implement a human-AI collaborative annotation-validation pipeline. DocPuzzle introduces an innovative evaluation framework that mitigates guessing bias through checklist-guided process analysis, establishing new standards for assessing reasoning capacities in LLMs. Our evaluation results show that: 1)Advanced slow-thinking reasoning models like o1-preview(69.7%) and DeepSeek-R1(66.3%) significantly outperform best general instruct models like Claude 3.5 Sonnet(57.7%); 2)Distilled reasoning models like DeepSeek-R1-Distill-Qwen-32B(41.3%) falls far behind the teacher model, suggesting challenges to maintain the generalization of reasoning capabilities relying solely on distillation.


Enhancing Human Evaluation in Machine Translation with Comparative Judgment

arXiv.org Artificial Intelligence

Human evaluation is crucial for assessing rapidly evolving language models but is influenced by annotator proficiency and task design. This study explores the integration of comparative judgment into human annotation for machine translation (MT) and evaluates three annotation setups-point-wise Multidimensional Quality Metrics (MQM), side-by-side (SxS) MQM, and its simplified version SxS relative ranking (RR). In MQM, annotators mark error spans with categories and severity levels. SxS MQM extends MQM to pairwise error annotation for two translations of the same input, while SxS RR focuses on selecting the better output without labeling errors. Key findings are: (1) the SxS settings achieve higher inter-annotator agreement than MQM; (2) SxS MQM enhances inter-translation error marking consistency compared to MQM by, on average, 38.5% for explicitly compared MT systems and 19.5% for others; (3) all annotation settings return stable system rankings, with SxS RR offering a more efficient alternative to (SxS) MQM; (4) the SxS settings highlight subtle errors overlooked in MQM without altering absolute system evaluations. To spur further research, we will release the triply annotated datasets comprising 377 ZhEn and 104 EnDe annotation examples.


Applications of deep reinforcement learning to urban transit network design

arXiv.org Artificial Intelligence

This thesis concerns the use of reinforcement learning to train neural networks to aid in the design of public transit networks. The Transit Network Design Problem (TNDP) is an optimization problem of considerable practical importance. Given a city with an existing road network and travel demands, the goal is to find a set of transit routes - each of which is a path through the graph - that collectively satisfy all demands, while minimizing a cost function that may depend both on passenger satisfaction and operating costs. The existing literature on this problem mainly considers metaheuristic optimization algorithms, such as genetic algorithms and ant-colony optimization. By contrast, we begin by taking a reinforcement learning approach, formulating the construction of a set of transit routes as a Markov Decision Process (MDP) and training a neural net policy to act as the agent in this MDP. We then show that, beyond using this policy to plan a transit network directly, it can be combined with existing metaheuristic algorithms, both to initialize the solution and to suggest promising moves at each step of a search through solution space. We find that such hybrid algorithms, which use a neural policy trained via reinforcement learning as a core component within a classical metaheuristic framework, can plan transit networks that are superior to those planned by either the neural policy or the metaheuristic algorithm. We demonstrate the utility of our approach by using it to redesign the transit network for the city of Laval, Quebec, and show that in simulation, the resulting transit network provides better service at lower cost than the existing transit network.


Phoeni6: a Systematic Approach for Evaluating the Energy Consumption of Neural Networks

arXiv.org Artificial Intelligence

This paper presents Phoeni6, a systematic approach for assessing the energy consumption of neural networks while upholding the principles of fair comparison and reproducibility. The methodology automates energy evaluations through containerized tools, robust database management, and versatile data models. In the first case study, the energy consumption of AlexNet and MobileNet was compared using raw and resized images. Results showed that MobileNet is up to 6.25% more energy-e fficient for raw images and 2.32% for resized datasets, while maintaining competitive accuracy levels. In the second study, the impact of image file formats on energy consumption was evaluated. BMP images reduced energy usage by up to 30% compared to PNG, highlighting the influence of file formats on energy e fficiency. These findings emphasize the importance of Phoeni6 in optimizing energy consumption for diverse neural network applications and establishing sustainable artificial intelligence practices. Introduction Deep Neural Networks (DNN) are being used with relative success in fields such as computer vision and natural language processing) [1, 2]. A growing number of initiatives have been promoting the development of these networks to solve everyday problems, including optimizing resource allocation in energy-constrained environments like wireless sensor networks [3]. There are repositories [4, 5] with hundreds of networks created and made available in lists ordered by accuracy, which is the primary metric used to assess the quality of each network. Their results emphasize that the search for energy efficiency can significantly benefit mobile devices' autonomy and positively a ff ect the financial costs and carbon footprints of large data centers distributed worldwide. These works measure energy to evaluate their technique. There is an evident global concern for the energy consumption of software products that a ffect people's daily lives--neural networks are becoming one of them. This fact has important implications on the criteria used to choose these products. It is reasonable to say that energy consumption is becoming part of the criteria for selecting neural networks, just as accuracy is. However, unlike the accuracy calculation, which fundamentally depends on the dataset and the procedures used during the training phase, the energy calculation depends on the devices involved. This aspect adds extra challenges to reproducing the results (RR) and making fair comparisons (FC) between di ff er-ent networks [24]. Evaluating the energy consumption of neural networks while adhering to the principles of Fair Comparison (FC) and Result Reproducibility (RR) presents significant challenges.


Bridging Information Gaps with Comprehensive Answers: Improving the Diversity and Informativeness of Follow-Up Questions

arXiv.org Artificial Intelligence

Effective conversational systems are expected to dynamically generate contextual follow-up questions to elicit new information while maintaining the conversation flow. While humans excel at asking diverse and informative questions by intuitively assessing both obtained and missing information, existing models often fall short of human performance on this task. To mitigate this, we propose a method that generates diverse and informative questions based on targeting unanswered information using a hypothetical LLM-generated "comprehensive answer". Our method is applied to augment an existing follow-up questions dataset. The experimental results demonstrate that language models fine-tuned on the augmented datasets produce follow-up questions of significantly higher quality and diversity. This promising approach could be effectively adopted to future work to augment information-seeking dialogues for reducing ambiguities and improving the accuracy of LLM answers.


Mind the Gesture: Evaluating AI Sensitivity to Culturally Offensive Non-Verbal Gestures

arXiv.org Artificial Intelligence

Gestures are an integral part of non-verbal communication, with meanings that vary across cultures, and misinterpretations that can have serious social and diplomatic consequences. As AI systems become more integrated into global applications, ensuring they do not inadvertently perpetuate cultural offenses is critical. To this end, we introduce Multi-Cultural Set of Inappropriate Gestures and Nonverbal Signs (MC-SIGNS), a dataset of 288 gesture-country pairs annotated for offensiveness, cultural significance, and contextual factors across 25 gestures and 85 countries. Through systematic evaluation using MC-SIGNS, we uncover critical limitations: text-to-image (T2I) systems exhibit strong US-centric biases, performing better at detecting offensive gestures in US contexts than in non-US ones; large language models (LLMs) tend to over-flag gestures as offensive; and vision-language models (VLMs) default to US-based interpretations when responding to universal concepts like wishing someone luck, frequently suggesting culturally inappropriate gestures. These findings highlight the urgent need for culturally-aware AI safety mechanisms to ensure equitable global deployment of AI technologies.