Law
Data-Centric Governance
McGregor, Sean, Hostetler, Jesse
Artificial intelligence (AI) governance is the body of standards and practices used to ensure that AI systems are deployed responsibly. Current AI governance approaches consist mainly of manual review and documentation processes. While such reviews are necessary for many systems, they are not sufficient to systematically address all potential harms, as they do not operationalize governance requirements for system engineering, behavior, and outcomes in a way that facilitates rigorous and reproducible evaluation. Modern AI systems are data-centric: they act on data, produce data, and are built through data engineering. The assurance of governance requirements must also be carried out in terms of data. This work explores the systematization of governance requirements via datasets and algorithmic evaluations. When applied throughout the product lifecycle, data-centric governance decreases time to deployment, increases solution quality, decreases deployment risks, and places the system in a continuous state of assured compliance with governance requirements.
A Review of the Role of Causality in Developing Trustworthy AI Systems
Ganguly, Niloy, Fazlija, Dren, Badar, Maryam, Fisichella, Marco, Sikdar, Sandipan, Schrader, Johanna, Wallat, Jonas, Rudra, Koustav, Koubarakis, Manolis, Patro, Gourab K., Amri, Wadhah Zai El, Nejdl, Wolfgang
As a result, they are often brittle and unable to adapt to new domains, can treat individuals or subgroups unfairly, and have limited ability to explain their actions or recommendations [197, 235] reducing the trust of human users [118]. Following this, a new area of research, trustworthy AI, has recently received much attention from several policymakers and other regulatory organizations. The resulting guidelines (e.g., [184, 186, 187]), introduced to increase trust in AI systems, make developing trustworthy AI not only a technical (research) and social endeavor but also an organizational and (legal) obligational requirement. In this paper, we set out to demonstrate, through an extensive survey, that causal modeling and reasoning is an emerging and very useful tool for enabling current AI systems to become trustworthy. Causality is the science of reasoning about causes and effects. Cause-and-effect relationships are central to how we make sense of the world around us, how we act upon it, and how we respond to changes in our environment. In AI, research in causality was pioneered by the Turing award winner Judea Pearl long back in his 1995 seminal paper [194]. Since then, many researchers have contributed to the development of a solid mathematical basis for causality; see, for example, the books [79, 196, 201], the survey [90] and seminal papers [197, 235].
Invisible Users: Uncovering End-Users' Requirements for Explainable AI via Explanation Forms and Goals
Jin, Weina, Fan, Jianyu, Gromala, Diane, Pasquier, Philippe, Hamarneh, Ghassan
Non-technical end-users are silent and invisible users of the state-of-the-art explainable artificial intelligence (XAI) technologies. Their demands and requirements for AI explainability are not incorporated into the design and evaluation of XAI techniques, which are developed to explain the rationales of AI decisions to end-users and assist their critical decisions. This makes XAI techniques ineffective or even harmful in high-stakes applications, such as healthcare, criminal justice, finance, and autonomous driving systems. To systematically understand end-users' requirements to support the technical development of XAI, we conducted the EUCA user study with 32 layperson participants in four AI-assisted critical tasks. The study identified comprehensive user requirements for feature-, example-, and rule-based XAI techniques (manifested by the end-user-friendly explanation forms) and XAI evaluation objectives (manifested by the explanation goals), which were shown to be helpful to directly inspire the proposal of new XAI algorithms and evaluation metrics. The EUCA study findings, the identified explanation forms and goals for technical specification, and the EUCA study dataset support the design and evaluation of end-user-centered XAI techniques for accessible, safe, and accountable AI.
A Brief Report on LawGPT 1.0: A Virtual Legal Assistant Based on GPT-3
LawGPT 1.0 is a virtual legal assistant built on the state-of-the-art language model GPT-3, fine-tuned for the legal domain. The system is designed to provide legal assistance to users in a conversational manner, helping them with tasks such as answering legal questions, generating legal documents, and providing legal advice. In this paper, we provide a brief overview of LawGPT 1.0, its architecture, and its performance on a set of legal benchmark tasks. Please note that the detailed information about the model is protected by a non-disclosure agreement (NDA) and cannot be disclosed in this report.
FedABC: Targeting Fair Competition in Personalized Federated Learning
Wang, Dui, Shen, Li, Luo, Yong, Hu, Han, Su, Kehua, Wen, Yonggang, Tao, Dacheng
Federated learning aims to collaboratively train models without accessing their client's local private data. The data may be Non-IID for different clients and thus resulting in poor performance. Recently, personalized federated learning (PFL) has achieved great success in handling Non-IID data by enforcing regularization in local optimization or improving the model aggregation scheme on the server. However, most of the PFL approaches do not take into account the unfair competition issue caused by the imbalanced data distribution and lack of positive samples for some classes in each client. To address this issue, we propose a novel and generic PFL framework termed Federated Averaging via Binary Classification, dubbed FedABC. In particular, we adopt the ``one-vs-all'' training strategy in each client to alleviate the unfair competition between classes by constructing a personalized binary classification problem for each class. This may aggravate the class imbalance challenge and thus a novel personalized binary classification loss that incorporates both the under-sampling and hard sample mining strategies is designed. Extensive experiments are conducted on two popular datasets under different settings, and the results demonstrate that our FedABC can significantly outperform the existing counterparts.
A Validity Perspective on Evaluating the Justified Use of Data-driven Decision-making Algorithms
Coston, Amanda, Kawakami, Anna, Zhu, Haiyi, Holstein, Ken, Heidari, Hoda
Recent research increasingly brings to question the appropriateness of using predictive tools in complex, real-world tasks. While a growing body of work has explored ways to improve value alignment in these tools, comparatively less work has centered concerns around the fundamental justifiability of using these tools. This work seeks to center validity considerations in deliberations around whether and how to build data-driven algorithms in high-stakes domains. Toward this end, we translate key concepts from validity theory to predictive algorithms. We apply the lens of validity to re-examine common challenges in problem formulation and data issues that jeopardize the justifiability of using predictive algorithms and connect these challenges to the social science discourse around validity. Our interdisciplinary exposition clarifies how these concepts apply to algorithmic decision making contexts. We demonstrate how these validity considerations could distill into a series of high-level questions intended to promote and document reflections on the legitimacy of the predictive task and the suitability of the data.
Big Tech Hasn't Fixed AI's Misinformation Problem--Yet
The scrappy underdog AI firm OpenAI has stirred the sleeping tech giants with its generative AI products, most recently and most prominently the conversational chatbot ChatGPT. Microsoft spent $10 billion on a partnership with OpenAI in an attempt to leap-frog its younger big tech competitors by weaving AI into many products; Google internally declared a "code red" and is cutting red tape to put out AI products more quickly, including a direct competitor to ChatGPT that was just announced; meanwhile Mark Zuckerberg has declared his intent to make Meta a "leader in generative AI," clearly a reaction to the attention OpenAI is garnering. The products these companies are suddenly striving for sound similar, but who will be the winner? Although much discussion has centered around the size of the AI models and how much data they are trained on, there's another factor that may matter a lot, too: the degree to which the contenders build trustworthy systems that don't unduly harm society and further destabilize democracy. OpenAI's earlier text generation product GPT-3 grabbed a lot of attention but never saw the widespread consumer adoption that ChatGPT has attained.
AI Generates Articles with Potentially Risky YMYL Content - Bytefeed - News Powered by AI
Artificial Intelligence (AI) is becoming increasingly popular in the world of content creation. AI-generated articles are now being used to create serious, Your Money or Your Life (YMYL) content for websites and other digital platforms. The use of AI-generated articles has been growing steadily over the past few years as more businesses recognize its potential to produce high quality, engaging content quickly and efficiently. AI can be used to generate both short form and long form pieces that cover a wide range of topics from finance and health care to travel and lifestyle. AI-generated YMYL content is particularly useful for businesses looking to provide accurate information on important topics such as financial advice, medical advice, legal advice or any other type of topic where accuracy is essential.
Does Your Current Use of AI in Financial Services Align with the U.S. "AI Bill of Rights"?
As OpenAI's release of ChatGPT in late 2022 and expected release of GPT-4 in 2023 continues to garner widespread attention, there is renewed focus on both opportunities and risks presented by the use of artificial intelligence ("AI"). With this focus comes the inevitable call for regulation. At the end of 2022, the U.S. White House weighed in through what it calls an "AI Bill of Rights" for the American public, a non-binding policy document. Banks and others in financial services should take note of the particular civil rights, privacy, and other priorities expressed in this vision for the future of AI governance. In financial services, technologies deploying some element of AI are expected to increase but already abound.
ChatGPT Is Passing the Tests Required for Medical Licenses and Business Degrees
Furthering its range of expertise, ChatGPT scored a 50 percent accuracy rate on the multiple-choice component of the Bar Exam, or the Multistate Bar Examination (MBE). The Bar Exam is the test that law school graduates need to pass in order to officially practice law and is composed of three parts, with the MBE being the first. GPT-3.5 reached the average passing rate for Torts and Evidence, which are two of the seven subject areas. The researchers concluded that due to these results, a large language model such as GPT will be able to "pass the MBE component of the Bar Exam in the near future."