Goto

Collaborating Authors

 Education


Tell Me More! Towards Implicit User Intention Understanding of Language Model Driven Agents

arXiv.org Artificial Intelligence

Current language model-driven agents often lack mechanisms for effective user participation, which is crucial given the vagueness commonly found in user instructions. Although adept at devising strategies and performing tasks, these agents struggle with seeking clarification and grasping precise user intentions. To bridge this gap, we introduce Intention-in-Interaction (IN3), a novel benchmark designed to inspect users' implicit intentions through explicit queries. Next, we propose the incorporation of model experts as the upstream in agent designs to enhance user-agent interaction. Employing IN3, we empirically train Mistral-Interact, a powerful model that proactively assesses task vagueness, inquires user intentions, and refines them into actionable goals before starting downstream agent task execution. Integrating it into the XAgent framework, we comprehensively evaluate the enhanced agent system regarding user instruction understanding and execution, revealing that our approach notably excels at identifying vague user tasks, recovering and summarizing critical missing information, setting precise and necessary agent execution goals, and minimizing redundant tool usage, thus boosting overall efficiency. All the data and codes are released.


Anti-gun activists use AI to recreate voices of mass shooting victims, taunt lawmakers with robocalls

FOX News

Families of gun violence victims are using artificial intelligence to recreate their loved ones' voices and taunt lawmakers who oppose gun control on the sixth anniversary of the Parkland massacre. The robocall messages are being sent to senators and House members who support the National Rifle Association and Second Amendment rights in a campaign that launched on Valentine's Day, Wednesday, according to the Associated Press. Manuel and Patricia Oliver, whose son Joaquin "Guac" Oliver died in the 2018 high school shooting in Parkland, Florida, said the campaign run through The Shotline website is intended to spur Congress to ban the sale of guns like the AR-15 rifle. "We come from a place where gun violence is a problem, but you will never see a 19-year-old with an AR-15 getting into a school and shooting people," Manuel Oliver told the Associated Press in an interview. The Olivers, immigrants from Venezuela, became activists after Joaquin and 13 other students at Marjory Stoneman Douglas High School were murdered by a 19-year-old killer with a rifle.


Voices of the dead: shooting victims plead for gun reform with AI-voice messages

The Guardian

Six years ago today, Joaquin Oliver was killed in a hallway outside his Florida classroom, one of 17 students and staff murdered in the worst high school shooting in the US. On Wednesday, lawmakers in Washington DC will hear his voice, recreated by artificial intelligence, in phone calls demanding to know why they've done nothing to tackle the plague of gun violence. "It's been six years and you've done nothing. Not a thing to stop all the shootings that have happened since," the message from Oliver, who was 17 when he died in the 2018 Valentine's Day's tragedy at Parkland's Marjory Stoneman Douglas high school, says. "I'm back today because my parents used AI to recreate my voice to call you. Other victims like me will be calling too, again and again, to demand action. How many calls will it take for you to care? How many dead voices will you hear before you finally listen?"


Newsom's top education advisor bares his mental health struggle: 'You're not alone'

Los Angeles Times

Six months into his first year in high school, he dropped out. For more than a year, he isolated himself in his Huntington Beach bedroom where he became addicted to video games and anonymously vented his anger online with racist and misogynistic screeds, haunted by suicidal thoughts and fantasies about hurting others. His health deteriorated as he binged on pepperoni pizza, grew obese and developed terrible rashes. Today, Chida, 38, is Gov. Gavin Newsom's chief deputy Cabinet secretary, a key member of the team building an ambitious plan to reshape public education through a 50-billion continuum of services to create a healthy foundation for children and a path to meaningful jobs at the end. Chida was the chief architect of five-year compacts with the University of California and California State University, pledging financial stability in exchange for gains in graduation rates, access and affordability.


A Semantic Social Network Analysis Tool for Sensitivity Analysis and What-If Scenario Testing in Alcohol Consumption Studies

arXiv.org Artificial Intelligence

Social Network Analysis (SNA) is a set of techniques developed in the field of social and behavioral sciences research, in order to characterize and study the social relationships that are established among a set of individuals. When building a social network for performing an SNA analysis, an initial process of data gathering is achieved in order to extract the characteristics of the individuals and their relationships. This is usually done by completing a questionnaire containing different types of questions that will be later used to obtain the SNA measures needed to perform the study. There are, then, a great number of different possible network generating questions and also many possibilities for mapping the responses to the corresponding characteristics and relationships. Many variations may be introduced into these questions (the way they are posed, the weights given to each of the responses, etc.) that may have an effect on the resulting networks. All these different variations are difficult to achieve manually, because the process is time-consuming and error prone. The tool described in this paper uses semantic knowledge representation techniques in order to facilitate this kind of sensitivity studies. The base of the tool is a conceptual structure, called "ontology" that is able to represent the different concepts and their definitions. The tool is compared to other similar ones, and the advantages of the approach are highlighted, giving some particular examples from an ongoing SNA study about alcohol consumption habits in adolescents.


FedSiKD: Clients Similarity and Knowledge Distillation: Addressing Non-i.i.d. and Constraints in Federated Learning

arXiv.org Artificial Intelligence

In recent years, federated learning (FL) has emerged as a promising technique for training machine learning models in a decentralized manner while also preserving data privacy. The non-independent and identically distributed (non-i.i.d.) nature of client data, coupled with constraints on client or edge devices, presents significant challenges in FL. Furthermore, learning across a high number of communication rounds can be risky and potentially unsafe for model exploitation. Traditional FL approaches may suffer from these challenges. Therefore, we introduce FedSiKD, which incorporates knowledge distillation (KD) within a similarity-based federated learning framework. As clients join the system, they securely share relevant statistics about their data distribution, promoting intra-cluster homogeneity. This enhances optimization efficiency and accelerates the learning process, effectively transferring knowledge between teacher and student models and addressing device constraints. FedSiKD outperforms state-of-the-art algorithms by achieving higher accuracy, exceeding by 25\% and 18\% for highly skewed data at $\alpha = {0.1,0.5}$ on the HAR and MNIST datasets, respectively. Its faster convergence is illustrated by a 17\% and 20\% increase in accuracy within the first five rounds on the HAR and MNIST datasets, respectively, highlighting its early-stage learning proficiency. Code is publicly available and hosted on GitHub (https://github.com/SimuEnv/FedSiKD)


Better-than-KL PAC-Bayes Bounds

arXiv.org Machine Learning

Let $f(\theta, X_1),$ $ \dots,$ $ f(\theta, X_n)$ be a sequence of random elements, where $f$ is a fixed scalar function, $X_1, \dots, X_n$ are independent random variables (data), and $\theta$ is a random parameter distributed according to some data-dependent posterior distribution $P_n$. In this paper, we consider the problem of proving concentration inequalities to estimate the mean of the sequence. An example of such a problem is the estimation of the generalization error of some predictor trained by a stochastic algorithm, such as a neural network where $f$ is a loss function. Classically, this problem is approached through a PAC-Bayes analysis where, in addition to the posterior, we choose a prior distribution which captures our belief about the inductive bias of the learning problem. Then, the key quantity in PAC-Bayes concentration bounds is a divergence that captures the complexity of the learning problem where the de facto standard choice is the KL divergence. However, the tightness of this choice has rarely been questioned. In this paper, we challenge the tightness of the KL-divergence-based bounds by showing that it is possible to achieve a strictly tighter bound. In particular, we demonstrate new high-probability PAC-Bayes bounds with a novel and better-than-KL divergence that is inspired by Zhang et al. (2022). Our proof is inspired by recent advances in regret analysis of gambling algorithms, and its use to derive concentration inequalities. Our result is first-of-its-kind in that existing PAC-Bayes bounds with non-KL divergences are not known to be strictly better than KL. Thus, we believe our work marks the first step towards identifying optimal rates of PAC-Bayes bounds.


Towards Privacy-Aware Sign Language Translation at Scale

arXiv.org Artificial Intelligence

A major impediment to the advancement of sign language translation (SLT) is data scarcity. Much of the sign language data currently available on the web cannot be used for training supervised models due to the lack of aligned captions. Furthermore, scaling SLT using large-scale web-scraped datasets bears privacy risks due to the presence of biometric information, which the responsible development of SLT technologies should account for. In this work, we propose a two-stage framework for privacy-aware SLT at scale that addresses both of these issues. We introduce SSVP-SLT, which leverages self-supervised video pretraining on anonymized and unannotated videos, followed by supervised SLT finetuning on a curated parallel dataset. SSVP-SLT achieves state-of-the-art finetuned and zero-shot gloss-free SLT performance on the How2Sign dataset, outperforming the strongest respective baselines by over 3 BLEU-4. Based on controlled experiments, we further discuss the advantages and limitations of self-supervised pretraining and anonymization via facial obfuscation for SLT.


LogicPrpBank: A Corpus for Logical Implication and Equivalence

arXiv.org Artificial Intelligence

Logic reasoning has been critically needed in problem-solving and decision-making. Although Language Models (LMs) have demonstrated capabilities of handling multiple reasoning tasks (e.g., commonsense reasoning), their ability to reason complex mathematical problems, specifically propositional logic, remains largely underexplored. This lack of exploration can be attributed to the limited availability of annotated corpora. Here, we present a well-labeled propositional logic corpus, LogicPrpBank, containing 7093 Propositional Logic Statements (PLSs) across six mathematical subjects, to study a brand-new task of reasoning logical implication and equivalence. We benchmark LogicPrpBank with widely-used LMs to show that our corpus offers a useful resource for this challenging task and there is ample room for model improvement.


Parametric Learning of Time-Advancement Operators for Unstable Flame Evolution

arXiv.org Artificial Intelligence

This study investigates the application of machine learning, specifically Fourier Neural Operator (FNO) and Convolutional Neural Network (CNN), to learn time-advancement operators for parametric partial differential equations (PDEs). Our focus is on extending existing operator learning methods to handle additional inputs representing PDE parameters. The goal is to create a unified learning approach that accurately predicts short-term solutions and provides robust long-term statistics under diverse parameter conditions, facilitating computational cost savings and accelerating development in engineering simulations. We develop and compare parametric learning methods based on FNO and CNN, evaluating their effectiveness in learning parametric-dependent solution time-advancement operators for one-dimensional PDEs and realistic flame front evolution data obtained from direct numerical simulations of the Navier-Stokes equations.