Goto

Collaborating Authors

 newcomer


Towards Robust Model Evolution with Algorithmic Recourse

Yang, Hao-Tsung, Gao, Jie, Liu, Bo-Yi, Liu, Zhi-Xuan

arXiv.org Artificial Intelligence

Algorithmic Recourse is a way for users to modify their attributes to align with a model's expectations, thereby improving their outcomes after receiving unfavorable decisions. In real-world scenarios, users often need to strategically adjust their attributes to compete for limited resources. However, such strategic behavior induces users to "game" algorithms, causing model collapse due to distribution shifts. These shifts arise from user competition, resource constraints, and adaptive user responses. While prior research on Algorithmic Recourse has explored its effects on both systems and users, the impact of resource constraints and competition over time remains underexplored. In this work, we develop a general framework to model user strategic behaviors and their interactions with decision-making systems under resource constraints and competitive dynamics. Through theoretical analysis and empirical evaluation, we identify three key phenomena that arise consistently in both synthetic and real-world datasets: escalating decision boundaries, non-robust model predictions, and inequitable recourse actions. Finally, we discuss the broader social implications of these findings and present two algorithmic strategies aimed at mitigating these challenges.


Examining the Prevalence and Dynamics of AI-Generated Media in Art Subreddits

Matatov, Hana, Quéré, Marianne Aubin Le, Amir, Ofra, Naaman, Mor

arXiv.org Artificial Intelligence

Broadly accessible generative AI models like Dall-E have made it possible for anyone to create compelling visual art. In online communities, the introduction of AI-generated content (AIGC) may impact community dynamics by shifting the kinds of content being posted or the responses to content suspected of being generated by AI. We take steps towards examining the potential impact of AIGC on art-related communities on Reddit. We distinguish between communities that disallow AI content and those without a direct policy. We look at image-based posts made to these communities that are transparently created by AI, or comments in these communities that suspect authors of using generative AI. We find that AI posts (and accusations) have played a very small part in these communities through the end of 2023, accounting for fewer than 0.2% of the image-based posts. Even as the absolute number of author-labelled AI posts dwindles over time, accusations of AI use remain more persistent. We show that AI content is more readily used by newcomers and may help increase participation if it aligns with community rules. However, the tone of comments suspecting AI use by others have become more negative over time, especially in communities that do not have explicit rules about AI. Overall, the results show the changing norms and interactions around AIGC in online communities designated for creativity.


Social and Ethical Risks Posed by General-Purpose LLMs for Settling Newcomers in Canada

Nejadgholi, Isar, Molamohammadi, Maryam, Bakhtawar, Samir

arXiv.org Artificial Intelligence

The non-profit settlement sector in Canada supports newcomers in achieving successful integration. This sector faces increasing operational pressures amidst rising immigration targets, which highlights a need for enhanced efficiency and innovation, potentially through reliable AI solutions. The ad-hoc use of general-purpose generative AI, such as ChatGPT, might become a common practice among newcomers and service providers to address this need. However, these tools are not tailored for the settlement domain and can have detrimental implications for immigrants and refugees. We explore the risks that these tools might pose on newcomers to first, warn against the unguarded use of generative AI, and second, to incentivize further research and development in creating AI literacy programs as well as customized LLMs that are aligned with the preferences of the impacted communities. Crucially, such technologies should be designed to integrate seamlessly into the existing workflow of the settlement sector, ensuring human oversight, trustworthiness, and accountability.


Supporting the Digital Autonomy of Elders Through LLM Assistance

Roberts, Jesse, Roberts, Lindsey, Reed, Alice

arXiv.org Artificial Intelligence

The internet offers tremendous access to services, social connections, and needed products. However, to those without sufficient experience, engaging with businesses and friends across the internet can be daunting due to the ever present danger of scammers and thieves, to say nothing of the myriad of potential computer viruses. Like a forest rich with both edible and poisonous plants, those familiar with the norms inhabit it safely with ease while newcomers need a guide. However, reliance on a human digital guide can be taxing and often impractical. We propose and pilot a simple but unexplored idea: could an LLM provide the necessary support to help the elderly who are separated by the digital divide safely achieve digital autonomy?


Double-Anonymous Review for Robotics

Yim, Justin K., Nadan, Paul, Zhu, James, Stutt, Alexandra, Payne, J. Joe, Pavlov, Catherine, Johnson, Aaron M.

arXiv.org Artificial Intelligence

However, Prior research has investigated the benefits and costs of even when reviewers self-report as having the highest level double-anonymous review (DAR, also known as double-blind of expertise in their field, their guess accuracy is no better review) in comparison to single-anonymous review (SAR) and than those who are self-reported as less knowledgeable [17]. Several review papers have attempted to Increased editor burden in handling conflict of interest, author compile experimental results in peer review research both burden in anonymizing the manuscript, and reviewer burden broadly and in engineering and computer science specifically in navigating prior work by others and by the authors are also [1-4]. This document summarizes prior research in peer review cited as costs to DAR. that may inform decisions about the format of peer review in Despite these challenges, numerous robotics conferences the field of robotics and makes some recommendations for have already made the shift to DAR, including RSS and a potential next steps for robotics publications. Furthermore, top machine learning conferences such as NeurIPS and CoRL have II. The presence of gender bias and effect of DAR on such bias is a common concern in research into peer review but Based on the current literature, we find that the evidence the conclusions are varied. Many studies do conclude that in support of double-anonymous review is not sufficient to gender can disadvantage authors, particularly women [5, 6] conclusively recommend for implementation in robotics conferences and that DAR can reduce this bias [7].


GiveMeLabeledIssues: An Open Source Issue Recommendation System

Vargovich, Joseph, Santos, Fabio, Penney, Jacob, Gerosa, Marco A., Steinmacher, Igor

arXiv.org Artificial Intelligence

Developers often struggle to navigate an Open Source Software (OSS) project's issue-tracking system and find a suitable task. Proper issue labeling can aid task selection, but current tools are limited to classifying the issues according to their type (e.g., bug, question, good first issue, feature, etc.). In contrast, this paper presents a tool (GiveMeLabeledIssues) that mines project repositories and labels issues based on the skills required to solve them. We leverage the domain of the APIs involved in the solution (e.g., User Interface (UI), Test, Databases (DB), etc.) as a proxy for the required skills. GiveMeLabeledIssues facilitates matching developers' skills to tasks, reducing the burden on project maintainers. The tool obtained a precision of 83.9% when predicting the API domains involved in the issues. The replication package contains instructions on executing the tool and including new projects. A demo video is available at https://www.youtube.com/watch?v=ic2quUue7i8


Supporting the Task-driven Skill Identification in Open Source Project Issue Tracking Systems

Santos, Fabio

arXiv.org Artificial Intelligence

Selecting an appropriate task is challenging for contributors to Open Source Software (OSS), mainly for those who are contributing for the first time. Therefore, researchers and OSS projects have proposed various strategies to aid newcomers, including labeling tasks. We investigate the automatic labeling of open issues strategy to help the contributors to pick a task to contribute. We label the issues with API-domains--categories of APIs parsed from the source code used to solve the issues. We plan to add social network analysis metrics from the issues conversations as new predictors. By identifying the skills, we claim the contributor candidates should pick a task more suitable. We analyzed interview transcripts and the survey's open-ended questions to comprehend the strategies used to assist in onboarding contributors and used to pick up an issue. We applied quantitative studies to analyze the relevance of the labels in an experiment and compare the strategies' relative importance. We also mined issue data from OSS repositories to predict the API-domain labels with comparable precision, recall, and F-measure with the state-of-art. We plan to use a skill ontology to assist the matching process between contributors and tasks. By analyzing the confidence level of the matching instances in ontologies describing contributors' skills and tasks, we might recommend issues for contribution. So far, the results showed that organizing the issues--which includes assigning labels is seen as an essential strategy for diverse roles in OSS communities. The API-domain labels are relevant for experienced practitioners. The predictions have an average precision of 75.5%. Labeling the issues indicates the skills involved in an issue. The labels represent possible skills in the source code related to an issue. By investigating this research topic, we expect to assist the new contributors in finding a task.


A newcomer's guide to #ICRA2022: Tutorials

Robohub

I believe that one of the best ways to get the training you need for a job market in robotics is to attend tutorials at conferences like ICRA. Unlike workshops where you might listen to some work-in-progress, other workshop paper presentations and panel discussions, tutorials are exactly what they sound like. They aim to give you some hands-on learning sessions on technical tools/skills with specific learning objectives. As such, most tutorials would expect you to come prepared to actively participate and follow along. For instance, the "Tools for Robotic Reinforcement Learning" tutorial expects you to come knowing how to code in python and have basic knowledge of reinforcement learning because you'll be expected to use those skills/knowledge in the hands-on sessions. There are seven tutorials this year.


PyCaret for Classification: An Honest Review

#artificialintelligence

Well, I had to do some quick ML work and wanted to try out something fairly new. I've seen PyCaret going around so I had to give it a try. PyCaret is a low-code open-source machine learning library for Python. It basically wraps a bunch of other libraries such as sklearn and xgboost and makes it super easy to try a lot of different models, blend them, stack them and stir the pot until something good comes out. It requires very little code to get from 0 to hero.


Updates and Lessons from AI Forecasting

#artificialintelligence

Earlier this year, my research group commissioned 6 questions for professional forecasters to predict about AI. They have financial incentives to produce accurate forecasts; the rewards total \$5k per question (\$30k total) and payoffs are (close to) a proper scoring rule, meaning forecasters are rewarded for outputting calibrated probabilities. You're in luck, because I'm going to answer each of these in the following sections! Feel free to skim to the ones that interest you the most. The particular questions were designed by my students Alex Wei, Collin Burns, Jean-Stanislas Denain, and Dan Hendrycks.