Law
A Critical Survey on Fairness Benefits of XAI
Deck, Luca, Schoeffer, Jakob, De-Arteaga, Maria, Kühl, Niklas
In this critical survey, we analyze typical claims on the relationship between explainable AI (XAI) and fairness to disentangle the multidimensional relationship between these two concepts. Based on a systematic literature review and a subsequent qualitative content analysis, we identify seven archetypal claims from 175 papers on the alleged fairness benefits of XAI. We present crucial caveats with respect to these claims and provide an entry point for future discussions around the potentials and limitations of XAI for specific fairness desiderata. Importantly, we notice that claims are often (i) vague and simplistic, (ii) lacking normative grounding, or (iii) poorly aligned with the actual capabilities of XAI. We encourage to conceive XAI not as an ethical panacea but as one of many tools to approach the multidimensional, sociotechnical challenge of algorithmic fairness. Moreover, when making a claim about XAI and fairness, we emphasize the need to be more specific about what kind of XAI method is used and which fairness desideratum it refers to, how exactly it enables fairness, and who is the stakeholder that benefits from XAI.
OpenAI Cribbed Our Tax Example, But Can GPT-4 Really Do Tax?
Blair-Stanek, Andrew, Holzenberger, Nils, Van Durme, Benjamin
The presenter pasted in what he called "about 16 pages' worth of tax code" These seven sentences about Alice, Bob, and Charlie come word-for-word from a handcrafted data set we developed at Johns Hopkins University and published in 2020 for training and measuring AI models for reasoning over statutory language. Every word, punctuation mark, and Maryland; Nils number in the taxpayer facts comes exactly from Holzenberger is an our tax_case_9 -- even the percent sign at the start associate professor in of the line. This work has been supported by the U.S. National Science Foundation under grant No. 2204926. The entire livestream is available at OpenAI, "GPT-4 Developer The tax law example starts at minute 19:11. Go to the directory "Cases" to find the file tax_case_9.pl. Tax_case_9.pl is written in the programming language Prolog. Federal content, please visit www.taxnotes.com. Where did the "about 16 pages' worth of tax out the TCJA standard deduction increase at code" come from? Again, from our 2020 data set. SARA has two deduction for 2018 was $24,000. From minute 20:07 to 20:40 of the livestream, handcrafted cases in SARA; tax_case_9 is one of we see some of the tax sections pasted into GPT-4. The statutes consist of nine sections of the These are SARA's heavily edited version of the IRC, For example, at and remove ambiguity. If you put all the SARA 20:23, we see part of section 63(c) with the statutes into a single file it will be about 16 pages paragraphs jumping from (3) to (5); in SARA, we long (depending on the font). At 20:26, we see part of section One of our edits was paring section 1 down to 63(c)(6) with only subparagraphs (A), (B), and (D); only sections 1(a) through (d), which contain the in SARA, we edited out (C). At 20:40, we see parts Clinton-era tax rates. We cut section 1(j), which of section 3306(b) with the paragraphs jumping contains the reduced Tax Cuts and Jobs Act rates from (2) to (7); in SARA, we edited out paragraphs for 2018-2025. This editing explains why GPT-4 (3) through (6). At 20:39 we see sections 3301 and got the wrong answer on the livestream for Alice 3306 regarding the federal unemployment tax; and Bob's 2018 taxes. We did not, however, edit while these two sections are irrelevant to Alice and Bob's tax liability in tax_case_9, they are two The author Holzenberger did all the handcrafting and hand editing. Federal content, please visit www.taxnotes.com. You can We empirically verified that using the SARA download our data set and compare it with the version of the IRC causes GPT-4 to get the wrong livestream's recording on YouTube. First, we The presenter then gives directions to GPT-4: pasted into GPT-4 all nine SARA statutes, plus our "Now calculate their total liability." GPT-4 gives facts about Alice, Bob, and Charlie. Then we detailed step-by-step calculations and concludes used the same "Now calculate their total liability" that "Alice and Bob's total tax liability for 2018 is command.
Phony AI Biden robocalls reached up to 25,000 voters, says New Hampshire AG
Two companies based in Texas have been linked to a spate of robocalls that used artificial intelligence to mimic President Joe Biden. The audio deepfake was used to urge New Hampshire voters not to participate in the state's presidential primary. New Hampshire Attorney General John Formella said as many as 25,000 of the calls were made to residents of the state in January. Formella says an investigation has linked the source of the robocalls to Texan companies Life Corporation and Lingo Telecom. No charges have yet been filed against either company or Life Corporation's owner, a person named Walter Monk.
TechScape: Why is the UK so slow to regulate AI?
Britain wants to lead the world in AI regulation. But AI regulation is a rapidly evolving, contested policy space in which there's little agreement over what a good outcome would look like, let alone the best methods to get there. And being the third most important hub of AI research in the world doesn't give you an awful lot of power when the first two are the US and China. How to slice through this Gordian knot? Simple: move swiftly and decisively to do … absolutely nothing.
Who benefits when shoppers use Amazon's new AI tool?
Amazon has begun rolling out a new artificial intelligence assistant that is meant to address shoppers product questions, but the feature raises as many questions as it answers. Rufus, as the software is known, will help users, according to Amazon, by guiding them to the toaster ovens or dinosaur toys that best fit their needs. Yet Amazon has a history of steering customers toward products that most benefit Amazon, either because they are more profitable or are backed by advertising dollars, according to the Federal Trade Commission's pending 2023 antitrust lawsuit against Amazon. The FTC accused Amazon of "biasing Amazon's search results to preference Amazon's own products over ones that Amazon knows are of better quality." Further, the FTC alleges the Seattle firm operates a "pay-to-play" system, giving top billing for the products on which marketers were willing to spend the most.
Monitoring the evolution of antisemitic discourse on extremist social media using BERT
Mustafa, Raza Ul, Japkowicz, Nathalie
Racism and intolerance on social media contribute to a toxic online environment which may spill offline to foster hatred, and eventually lead to physical violence. That is the case with online antisemitism, the specific category of hatred considered in this study. Tracking antisemitic themes and their associated terminology over time in online discussions could help monitor the sentiments of their participants and their evolution, and possibly offer avenues for intervention that may prevent the escalation of hatred. Due to the large volume and constant evolution of online traffic, monitoring conversations manually is impractical. Instead, we propose an automated method that extracts antisemitic themes and terminology from extremist social media over time and captures their evolution. Since supervised learning would be too limited for such a task, we created an unsupervised online machine learning approach that uses large language models to assess the contextual similarity of posts. The method clusters similar posts together, dividing, and creating additional clusters over time when sub-themes emerge from existing ones or new themes appear. The antisemitic terminology used within each theme is extracted from the posts in each cluster. Our experiments show that our methodology outperforms existing baselines and demonstrates the kind of themes and sub-themes it discovers within antisemitic discourse along with their associated terminology. We believe that our approach will be useful for monitoring the evolution of all kinds of hatred beyond antisemitism on social platforms.
Unmasking the Shadows of AI: Investigating Deceptive Capabilities in Large Language Models
This research critically navigates the intricate landscape of AI deception, concentrating on deceptive behaviours of Large Language Models (LLMs). My objective is to elucidate this issue, examine the discourse surrounding it, and subsequently delve into its categorization and ramifications. The essay initiates with an evaluation of the AI Safety Summit 2023 (ASS) and introduction of LLMs, emphasising multidimensional biases that underlie their deceptive behaviours.The literature review covers four types of deception categorised: Strategic deception, Imitation, Sycophancy, and Unfaithful Reasoning, along with the social implications and risks they entail. Lastly, I take an evaluative stance on various aspects related to navigating the persistent challenges of the deceptive AI. This encompasses considerations of international collaborative governance, the reconfigured engagement of individuals with AI, proposal of practical adjustments, and specific elements of digital education.
Measuring machine learning harms from stereotypes: requires understanding who is being harmed by which errors in what ways
Wang, Angelina, Bai, Xuechunzi, Barocas, Solon, Blodgett, Su Lin
As machine learning applications proliferate, we need an understanding of their potential for harm. However, current fairness metrics are rarely grounded in human psychological experiences of harm. Drawing on the social psychology of stereotypes, we use a case study of gender stereotypes in image search to examine how people react to machine learning errors. First, we use survey studies to show that not all machine learning errors reflect stereotypes nor are equally harmful. Then, in experimental studies we randomly expose participants to stereotype-reinforcing, -violating, and -neutral machine learning errors. We find stereotype-reinforcing errors induce more experientially (i.e., subjectively) harmful experiences, while having minimal changes to cognitive beliefs, attitudes, or behaviors. This experiential harm impacts women more than men. However, certain stereotype-violating errors are more experientially harmful for men, potentially due to perceived threats to masculinity. We conclude that harm cannot be the sole guide in fairness mitigation, and propose a nuanced perspective depending on who is experiencing what harm and why.
Counterfactual Generation with Answer Set Programming
Dasgupta, Sopam, Shakerin, Farhad, Arias, Joaquín, Salazar, Elmer, Gupta, Gopal
Machine learning models that automate decision-making are increasingly being used in consequential areas such as loan approvals, pretrial bail approval, hiring, and many more. Unfortunately, most of these models are black-boxes, i.e., they are unable to reveal how they reach these prediction decisions. A need for transparency demands justification for such predictions. An affected individual might also desire explanations to understand why a decision was made. Ethical and legal considerations may further require informing the individual of changes in the input attribute that could be made to produce a desirable outcome. This paper focuses on the latter problem of automatically generating counterfactual explanations. We propose a framework Counterfactual Generation with s(CASP) (CFGS) that utilizes answer set programming (ASP) and the s(CASP) goal-directed ASP system to automatically generate counterfactual explanations from rules generated by rule-based machine learning (RBML) algorithms. In our framework, we show how counterfactual explanations are computed and justified by imagining worlds where some or all factual assumptions are altered/changed. More importantly, we show how we can navigate between these worlds, namely, go from our original world/scenario where we obtain an undesired outcome to the imagined world/scenario where we obtain a desired/favourable outcome.
Ten Hard Problems in Artificial Intelligence We Must Get Right
Leech, Gavin, Garfinkel, Simson, Yagudin, Misha, Briand, Alexander, Zhuravlev, Aleksandr
We explore the AI2050 "hard problems" that block the promise of AI and cause AI risks: (1) developing general capabilities of the systems; (2) assuring the performance of AI systems and their training processes; (3) aligning system goals with human goals; (4) enabling great applications of AI in real life; (5) addressing economic disruptions; (6) ensuring the participation of all; (7) at the same time ensuring socially responsible deployment; (8) addressing any geopolitical disruptions that AI causes; (9) promoting sound governance of the technology; and (10) managing the philosophical disruptions for humans living in the age of AI. For each problem, we outline the area, identify significant recent work, and suggest ways forward. [Note: this paper reviews literature through January 2023.]