Law
Human-centered NLP Fact-checking: Co-Designing with Fact-checkers using Matchmaking for AI
Liu, Houjiang, Das, Anubrata, Boltz, Alexander, Zhou, Didi, Pinaroc, Daisy, Lease, Matthew, Lee, Min Kyung
A key challenge in professional fact-checking is its limited scalability in relation to the magnitude of false information. While many Natural Language Processing (NLP) tools have been proposed to enhance fact-checking efficiency and scalability, both academic research and fact-checking organizations report limited adoption of such tooling due to insufficient alignment with fact-checker practices, values, and needs. To address this gap, we investigate a co-design method, Matchmaking for AI, which facilitates fact-checkers, designers, and NLP researchers to collaboratively discover what fact-checker needs should be addressed by technology and how. Our co-design sessions with 22 professional fact-checkers yielded a set of 11 novel design ideas. They assist in information searching, processing, and writing tasks for efficient and personalized fact-checking; help fact-checkers proactively prepare for future misinformation; monitor their potential biases; and support internal organization collaboration. Our work offers implications for human-centered fact-checking research and practice and AI co-design research.
Improving Zero-Shot Text Matching for Financial Auditing with Large Language Models
Hillebrand, Lars, Berger, Armin, Deußer, Tobias, Dilmaghani, Tim, Khaled, Mohamed, Kliem, Bernd, Loitz, Rüdiger, Pielka, Maren, Leonhard, David, Bauckhage, Christian, Sifa, Rafet
Auditing financial documents is a very tedious and time-consuming process. As of today, it can already be simplified by employing AI-based solutions to recommend relevant text passages from a report for each legal requirement of rigorous accounting standards. However, these methods need to be fine-tuned regularly, and they require abundant annotated data, which is often lacking in industrial environments. Hence, we present ZeroShotALI, a novel recommender system that leverages a state-of-the-art large language model (LLM) in conjunction with a domain-specifically optimized transformer-based text-matching solution. We find that a two-step approach of first retrieving a number of best matching document sections per legal requirement with a custom BERT-based model and second filtering these selections using an LLM yields significant performance improvements over existing approaches.
Generative Interpretation
Arbel, Yonathan A., Hoffman, David
We introduce generative interpretation, a new approach to estimating contractual meaning using large language models. As AI triumphalism is the order of the day, we proceed by way of grounded case studies, each illustrating the capabilities of these novel tools in distinct ways. Taking well-known contracts opinions, and sourcing the actual agreements that they adjudicated, we show that AI models can help factfinders ascertain ordinary meaning in context, quantify ambiguity, and fill gaps in parties' agreements. We also illustrate how models can calculate the probative value of individual pieces of extrinsic evidence. After offering best practices for the use of these models given their limitations, we consider their implications for judicial practice and contract theory. Using LLMs permits courts to estimate what the parties intended cheaply and accurately, and as such generative interpretation unsettles the current interpretative stalemate. Their use responds to efficiency-minded textualists and justice-oriented contextualists, who argue about whether parties will prefer cost and certainty or accuracy and fairness. Parties--and courts--would prefer a middle path, in which adjudicators strive to predict what the contract really meant, admitting just enough context to approximate reality while avoiding unguided and biased assimilation of evidence. As generative interpretation offers this possibility, we argue it can become the new workhorse of contractual interpretation.
Benign Shortcut for Debiasing: Fair Visual Recognition via Intervention with Shortcut Features
Zhang, Yi, Sang, Jitao, Wang, Junyang, Jiang, Dongmei, Wang, Yaowei
Machine learning models often learn to make predictions that rely on sensitive social attributes like gender and race, which poses significant fairness risks, especially in societal applications, such as hiring, banking, and criminal justice. Existing work tackles this issue by minimizing the employed information about social attributes in models for debiasing. However, the high correlation between target task and these social attributes makes learning on the target task incompatible with debiasing. Given that model bias arises due to the learning of bias features (\emph{i.e}., gender) that help target task optimization, we explore the following research question: \emph{Can we leverage shortcut features to replace the role of bias feature in target task optimization for debiasing?} To this end, we propose \emph{Shortcut Debiasing}, to first transfer the target task's learning of bias attributes from bias features to shortcut features, and then employ causal intervention to eliminate shortcut features during inference. The key idea of \emph{Shortcut Debiasing} is to design controllable shortcut features to on one hand replace bias features in contributing to the target task during the training stage, and on the other hand be easily removed by intervention during the inference stage. This guarantees the learning of the target task does not hinder the elimination of bias features. We apply \emph{Shortcut Debiasing} to several benchmark datasets, and achieve significant improvements over the state-of-the-art debiasing methods in both accuracy and fairness.
Man sentenced 16 years after sexually assaulting Minneapolis woman later injured in deadly car crash
Harvey Castro talks about how AI could be used in cold cases and the symbiotic relationship between AI and a detective. A Minnesota man was sentenced to 30 years in prison for raping a woman at gunpoint in Minneapolis 16 years ago. Robert DeLong, 63, will spend the next three decades in prison after he pleaded guilty to assaulting the victim who was jogging on Boom Island in Minneapolis in March 2007. Hennepin County Attorney Mary Moriarty's office told Fox News Digital that 30 years is the "longest possible sentence" for DeLong's crimes. "The victim's courage in the moments after this attack are a significant reason we were able to prosecute this case and hold Mr. DeLong accountable," Moriarty said in a statement.
A 'black box' AI system has been influencing criminal justice decisions for over two decades – it's time to open it up
Justice systems around the world are using artificial intelligence (AI) to assess people with criminal convictions. These AI technologies rely on machine learning algorithms and their key purpose is to predict the risk of reoffending. They influence decisions made by the courts and prisons and by parole and probation officers. This kind of tech has been an intrinsic part of the UK justice system since 2001. That was the year a risk assessment tool, known as Oasys (Offender Assessment System), was introduced and began taking over certain tasks from probation officers. Yet in over two decades, scientists outside the government have not been permitted access to the data behind Oasys to independently analyse its workings and assess its accuracy – for example, whether the decisions it influences lead to fewer offences or reconvictions. Lack of transparency affects AI systems generally. Their complex decision-making processes can evolve into a black box – too obscure to unravel without advanced technical knowledge. Proponents believe that AI algorithms are more objective scientific tools because they are standardised and this helps to reduce human bias in assessments and decision making. This, supporters claim, makes them useful for public protection. But critics say that a lack of access to the data, as well as other crucial information required for independent evaluation, raises serious questions of accountability and transparency.
The threatening potential of AI and child abuse
Canopy CMO Yaron Litwin discusses how criminals are using deepfake technology to blackmail teens and generate child pornography. Most Americans don't have a clue about artificial intelligence and what it means to the world's inhabitants. For those who are in this fog, the person who is a heartbeat away from the presidency has added her clarity to the mix. "I think the first part of this issue should be articulated is AI is a kind of a fancy thing, first of all, it's two letters, it means artificial intelligence but ultimately… it's machine learning." Now that Vice President Harris has defined artificial intelligence for us, she further enlightens our minds by elaborating, "And so, the machine is taught, and part of the issue here is what information is going into the machine that will then determine, and we can predict then if we think about what information is going in, what then will be produced in terms of decisions and opinions that may be made through that process."
Large Language Models and Knowledge Graphs: Opportunities and Challenges
Pan, Jeff Z., Razniewski, Simon, Kalo, Jan-Christoph, Singhania, Sneha, Chen, Jiaoyan, Dietze, Stefan, Jabeen, Hajira, Omeliyanenko, Janna, Zhang, Wen, Lissandrini, Matteo, Biswas, Russa, de Melo, Gerard, Bonifati, Angela, Vakaj, Edlira, Dragoni, Mauro, Graux, Damien
Large Language Models (LLMs) have taken Knowledge Representation -- and the world -- by storm. This inflection point marks a shift from explicit knowledge representation to a renewed focus on the hybrid representation of both explicit knowledge and parametric knowledge. In this position paper, we will discuss some of the common debate points within the community on LLMs (parametric knowledge) and Knowledge Graphs (explicit knowledge) and speculate on opportunities and visions that the renewed focus brings, as well as related research topics and challenges.
Software Doping Analysis for Human Oversight
Biewer, Sebastian, Baum, Kevin, Sterz, Sarah, Hermanns, Holger, Hetmank, Sven, Langer, Markus, Lauber-Rönsberg, Anne, Lehr, Franz
This article introduces a framework that is meant to assist in mitigating societal risks that software can pose. Concretely, this encompasses facets of software doping as well as unfairness and discrimination in high-risk decision-making systems. The term software doping refers to software that contains surreptitiously added functionality that is against the interest of the user. A prominent example of software doping are the tampered emission cleaning systems that were found in millions of cars around the world when the diesel emissions scandal surfaced. The first part of this article combines the formal foundations of software doping analysis with established probabilistic falsification techniques to arrive at a black-box analysis technique for identifying undesired effects of software. We apply this technique to emission cleaning systems in diesel cars but also to high-risk systems that evaluate humans in a possibly unfair or discriminating way. We demonstrate how our approach can assist humans-in-the-loop to make better informed and more responsible decisions. This is to promote effective human oversight, which will be a central requirement enforced by the European Union's upcoming AI Act. We complement our technical contribution with a juridically, philosophically, and psychologically informed perspective on the potential problems caused by such systems.
Generative AI Is Making Companies Even More Thirsty for Your Data
Zoom, the company that normalized attending business meetings in your pajama pants, was forced to unmute itself this week to reassure users that it would not use personal data to train artificial intelligence without their consent. A keen-eyed Hacker News user last week noticed that an update to Zoom's terms and conditions in March appeared to essentially give the company free rein to slurp up voice, video, and other data, and shovel it into machine learning systems. The new terms stated that customers "consent to Zoom's access, use, collection, creation, modification, distribution, processing, sharing, maintenance, and storage of Service Generated Data" for purposes including "machine learning or artificial intelligence (including for training and tuning of algorithms and models)." The discovery prompted critical news articles and angry posts across social media. On Monday, Zoom's chief product officer, Smita Hasham, wrote a blog post stating, "We will not use audio, video, or chat customer content to train our artificial intelligence models without your consent." The company also updated its terms to say the same.