Law
The VoicePrivacy 2024 Challenge Evaluation Plan
Tomashenko, Natalia, Miao, Xiaoxiao, Champion, Pierre, Meyer, Sarina, Wang, Xin, Vincent, Emmanuel, Panariello, Michele, Evans, Nicholas, Yamagishi, Junichi, Todisco, Massimiliano
The task of the challenge is to develop a voice anonymization system for speech data which conceals the speaker's voice identity while protecting linguistic content and emotional states. The organizers provide development and evaluation datasets and evaluation scripts, as well as baseline anonymization systems and a list of training resources formed on the basis of the participants' requests. Participants apply their developed anonymization systems, run evaluation scripts and submit evaluation results and anonymized speech data to the organizers. Results will be presented at a workshop held in conjunction with Interspeech 2024 to which all participants are invited to present their challenge systems and to submit additional workshop papers.
An Empirical Design Justice Approach to Identifying Ethical Considerations in the Intersection of Large Language Models and Social Robotics
The integration of Large Language Models (LLMs) in social robotics presents a unique set of ethical challenges and social impacts. This research is set out to identify ethical considerations that arise in the design and development of these two technologies in combination. Using LLMs for social robotics may provide benefits, such as enabling natural language open-domain dialogues. However, the intersection of these two technologies also gives rise to ethical concerns related to misinformation, non-verbal cues, emotional disruption, and biases. The robot's physical social embodiment adds complexity, as ethical hazards associated with LLM-based Social AI, such as hallucinations and misinformation, can be exacerbated due to the effects of physical embodiment on social perception and communication. To address these challenges, this study employs an empirical design justice-based methodology, focusing on identifying socio-technical ethical considerations through a qualitative co-design and interaction study. The purpose of the study is to identify ethical considerations relevant to the process of co-design of, and interaction with a humanoid social robot as the interface of a LLM, and to evaluate how a design justice methodology can be used in the context of designing LLMs-based social robotics. The findings reveal a mapping of ethical considerations arising in four conceptual dimensions: interaction, co-design, terms of service and relationship and evaluates how a design justice approach can be used empirically in the intersection of LLMs and social robotics.
Language Model Council: Benchmarking Foundation Models on Highly Subjective Tasks by Consensus
Zhao, Justin, Plaza-del-Arco, Flor Miriam, Curry, Amanda Cercas
The rapid advancement of Large Language Models (LLMs) necessitates robust and challenging benchmarks. Leaderboards like Chatbot Arena rank LLMs based on how well their responses align with human preferences. However, many tasks such as those related to emotional intelligence, creative writing, or persuasiveness, are highly subjective and often lack majoritarian human agreement. Judges may have irreconcilable disagreements about what constitutes a better response. To address the challenge of ranking LLMs on highly subjective tasks, we propose a novel benchmarking framework, the Language Model Council (LMC). The LMC operates through a democratic process to: 1) formulate a test set through equal participation, 2) administer the test among council members, and 3) evaluate responses as a collective jury. We deploy a council of 20 newest LLMs on an open-ended emotional intelligence task: responding to interpersonal dilemmas. Our results show that the LMC produces rankings that are more separable, robust, and less biased than those from any individual LLM judge, and is more consistent with a human-established leaderboard compared to other benchmarks.
Global AI Governance in Healthcare: A Cross-Jurisdictional Regulatory Analysis
Chakraborty, Attrayee, Karhade, Mandar
Artificial Intelligence (AI) is being adopted across the world and promises a new revolution in healthcare. While AI-enabled medical devices in North America dominate 42.3% of the global market, the use of AI-enabled medical devices in other countries is still a story waiting to be unfolded. We aim to delve deeper into global regulatory approaches towards AI use in healthcare, with a focus on how common themes are emerging globally. We compare these themes to the World Health Organization's (WHO) regulatory considerations and principles on ethical use of AI for healthcare applications. Our work seeks to take a global perspective on AI policy by analyzing 14 legal jurisdictions including countries representative of various regions in the world (North America, South America, South East Asia, Middle East, Africa, Australia, and the Asia-Pacific). Our eventual goal is to foster a global conversation on the ethical use of AI in healthcare and the regulations that will guide it. We propose solutions to promote international harmonization of AI regulations and examine the requirements for regulating generative AI, using China and Singapore as examples of countries with well-developed policies in this area.
Making AI Intelligible: Philosophical Foundations
Can humans and artificial intelligences share concepts and communicate? 'Making AI Intelligible' shows that philosophical work on the metaphysics of meaning can help answer these questions. Herman Cappelen and Josh Dever use the externalist tradition in philosophy to create models of how AIs and humans can understand each other. In doing so, they illustrate ways in which that philosophical tradition can be improved. The questions addressed in the book are not only theoretically interesting, but the answers have pressing practical implications. Many important decisions about human life are now influenced by AI. In giving that power to AI, we presuppose that AIs can track features of the world that we care about (for example, creditworthiness, recidivism, cancer, and combatants). If AIs can share our concepts, that will go some way towards justifying this reliance on AI. This ground-breaking study offers insight into how to take some first steps towards achieving Interpretable AI.
Fine-Tuned 'Small' LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification
Bucher, Martin Juan Josรฉ, Martini, Marco
Generative AI offers a simple, prompt-based alternative to fine-tuning smaller BERT-style LLMs for text classification tasks. This promises to eliminate the need for manually labeled training data and task-specific model training. However, it remains an open question whether tools like ChatGPT can deliver on this promise. In this paper, we show that smaller, fine-tuned LLMs (still) consistently and significantly outperform larger, zero-shot prompted models in text classification. We compare three major generative AI models (ChatGPT with GPT-3.5/GPT-4 and Claude Opus) with several fine-tuned LLMs across a diverse set of classification tasks (sentiment, approval/disapproval, emotions, party positions) and text categories (news, tweets, speeches). We find that fine-tuning with application-specific training data achieves superior performance in all cases. To make this approach more accessible to a broader audience, we provide an easy-to-use toolkit alongside this paper. Our toolkit, accompanied by non-technical step-by-step guidance, enables users to select and fine-tune BERT-like LLMs for any classification task with minimal technical and computational effort.
Elon Musk drops lawsuit against OpenAI
Musk originally filed his lawsuit at the beginning of March, arguing that OpenAI had breached its commitment to early investors and the public to build AI for the benefit of humanity when it began making money. At the time, OpenAI executives blamed Musk's lawsuit on his not being a part of the company as it was seeing massive success. Musk did not immediately respond to a request for comment Tuesday. A spokesperson for OpenAI declined to comment.
Elon Musk Drops Suit Accusing OpenAI of Breaching Founding Mission
Elon Musk dropped a lawsuit alleging OpenAI and its chief executive officer Sam Altman breached a founding promise last year by prioritizing profits over humanity. The billionaire withdrew his complaint a day before a California judge was set to hear OpenAI's request for dismissal. Musk had accused the company of becoming a "de facto subsidiary" of Microsoft Corp. in violation of a founding agreement to be a non-profit that developed artificial intelligence "for the benefit of humanity." OpenAI and Musk have been engaged in a well-publicized battle since well before the court case. Musk was an early backer of the startup and part of its founding team, before he had a falling out with the company.
Elon Musk abruptly withdraws lawsuit against Sam Altman and OpenAI
Elon Musk has moved to dismiss his lawsuit accusing ChatGPT maker OpenAI and its CEO Sam Altman of abandoning the startup's original mission of developing artificial intelligence for the benefit of humanity. Musk launched the suit against Altman in February, and the case had been slowly working its way through the California court system. There was no indication until Tuesday that Musk planned to drop the suit; only a month ago, his lawyers filed a challenge that forced the judge hearing the case to remove himself. Musk's request for a dismissal contained no reason behind the decision. A San Francisco superior court judge was scheduled on Wednesday to hear Altman and OpenAI's argument for throwing the case out.
Musk withdraws his breach of contract lawsuit against OpenAI
Elon Musk dropped a lawsuit against OpenAI one day before a judge in California state court was set to hear OpenAI's request for dismissal. Musk's suit, which was filed in February, had accused OpenAI co-founders Sam Altman and Greg Brockman of violating the company's non-profit status and instead prioritizing profits over using AI to help humanity. In the 35-page suit, Musk had alleged that OpenAI had become a "closed-source de facto subsidiary" of Microsoft, which invested 13 billion in the company and owns a 49 percent stake. Microsoft uses OpenAI's technology to power Copilot, the company's generative AI tools that are deeply integrated in products like Windows and Office. OpenAI had reportedly requested for the lawsuit to be dismissed, arguing that Musk would use any information that emerged as a result to get access to the company's "proprietary records and technology."