Law
The Surprising Effectiveness of Membership Inference with Simple N-Gram Coverage
Hallinan, Skyler, Jung, Jaehun, Sclar, Melanie, Lu, Ximing, Ravichander, Abhilasha, Ramnath, Sahana, Choi, Yejin, Karimireddy, Sai Praneeth, Mireshghallah, Niloofar, Ren, Xiang
Membership inference attacks serves as useful tool for fair use of language models, such as detecting potential copyright infringement and auditing data leakage. However, many current state-of-the-art attacks require access to models' hidden states or probability distribution, which prevents investigation into more widely-used, API-access only models like GPT-4. In this work, we introduce N-Gram Coverage Attack, a membership inference attack that relies solely on text outputs from the target model, enabling attacks on completely black-box models. We leverage the observation that models are more likely to memorize and subsequently generate text patterns that were commonly observed in their training data. Specifically, to make a prediction on a candidate member, N-Gram Coverage Attack first obtains multiple model generations conditioned on a prefix of the candidate. It then uses n-gram overlap metrics to compute and aggregate the similarities of these outputs with the ground truth suffix; high similarities indicate likely membership. We first demonstrate on a diverse set of existing benchmarks that N-Gram Coverage Attack outperforms other black-box methods while also impressively achieving comparable or even better performance to state-of-the-art white-box attacks - despite having access to only text outputs. Interestingly, we find that the success rate of our method scales with the attack compute budget - as we increase the number of sequences generated from the target model conditioned on the prefix, attack performance tends to improve. Having verified the accuracy of our method, we use it to investigate previously unstudied closed OpenAI models on multiple domains. We find that more recent models, such as GPT-4o, exhibit increased robustness to membership inference, suggesting an evolving trend toward improved privacy protections.
Collective dynamics of strategic classification
Couto, Marta C., Barsotti, Flavia, Santos, Fernando P.
Classification algorithms based on Artificial Intelligence (AI) are nowadays applied in high-stakes decisions in finance, healthcare, criminal justice, or education. Individuals can strategically adapt to the information gathered about classifiers, which in turn may require algorithms to be re-trained. Which collective dynamics will result from users' adaptation and algorithms' retraining? We apply evolutionary game theory to address this question. Our framework provides a mathematically rigorous way of treating the problem of feedback loops between collectives of users and institutions, allowing to test interventions to mitigate the adverse effects of strategic adaptation. As a case study, we consider institutions deploying algorithms for credit lending. We consider several scenarios, each representing different interaction paradigms. When algorithms are not robust against strategic manipulation, we are able to capture previous challenges discussed in the strategic classification literature, whereby users either pay excessive costs to meet the institutions' expectations (leading to high social costs) or game the algorithm (e.g., provide fake information). From this baseline setting, we test the role of improving gaming detection and providing algorithmic recourse. We show that increased detection capabilities reduce social costs and could lead to users' improvement; when perfect classifiers are not feasible (likely to occur in practice), algorithmic recourse can steer the dynamics towards high users' improvement rates. The speed at which the institutions re-adapt to the user's population plays a role in the final outcome. Finally, we explore a scenario where strict institutions provide actionable recourse to their unsuccessful users and observe cycling dynamics so far unnoticed in the literature.
Fairness of Automatic Speech Recognition: Looking Through a Philosophical Lens
Choi, Anna Seo Gyeong, Choi, Hoon
Automatic Speech Recognition (ASR) systems now mediate countless human-technology interactions, yet research on their fairness implications remains surprisingly limited. This paper examines ASR bias through a philosophical lens, arguing that systematic misrecognition of certain speech varieties constitutes more than a technical limitation -- it represents a form of disrespect that compounds historical injustices against marginalized linguistic communities. We distinguish between morally neutral classification (discriminate1) and harmful discrimination (discriminate2), demonstrating how ASR systems can inadvertently transform the former into the latter when they consistently misrecognize non-standard dialects. We identify three unique ethical dimensions of speech technologies that differentiate ASR bias from other algorithmic fairness concerns: the temporal burden placed on speakers of non-standard varieties ("temporal taxation"), the disruption of conversational flow when systems misrecognize speech, and the fundamental connection between speech patterns and personal/cultural identity. These factors create asymmetric power relationships that existing technical fairness metrics fail to capture. The paper analyzes the tension between linguistic standardization and pluralism in ASR development, arguing that current approaches often embed and reinforce problematic language ideologies. We conclude that addressing ASR bias requires more than technical interventions; it demands recognition of diverse speech varieties as legitimate forms of expression worthy of technological accommodation. This philosophical reframing offers new pathways for developing ASR systems that respect linguistic diversity and speaker autonomy.
Elon Musk and Sam Altman's AI Feud Gets Nasty
A long-running feud between Elon Musk and Sam Altman spilled out into the open this week as the AI billionaire heavyweights publicly fought over their rival companies. The latest round in the battle between the X CEO and the CEO of OpenAI began when Musk claimed that Apple had been favoring Altman's AI app over his own in the Apple Store rankings. "Apple is behaving in a manner that makes it impossible for any AI company besides OpenAI to reach #1 in the App Store, which is an unequivocal antitrust violation," Musk said on X on Monday evening. "xAI will take immediate legal action," he added, referring to the AI company he leads. "Hey @Apple App Store, why do you refuse to put either X or Grok in your'Must Have' section when X is the #1 news app in the world and Grok is #5 among all apps?" he asked.
Concerning the Responsible Use of AI in the U.S. Criminal Justice System
Artificial intelligence (AI) is advancing quickly and is being adopted in most industries. Using AI to draft an email message or check your grammar is typically not a cause for concern, but using it to make decisions that affect people's lives is another matter. When constitutional rights are involved, as in the justice system, transparency is paramount. During the Biden-Harris administration, Executive Order 14110 directed agencies to develop guidelines for acceptable uses and regulation of AI. Some of these uses, like summarizing and notetaking, will occur across the government.
Live facial recognition is 'worrying for our democracy', experts warn as the government expands the 'Orwellian' system across Britain
Experts have warned of a'frightening expansion' of'Orwellian' technology as the government expands the use of live facial recognition across the country. Ten vans equipped with facial recognition cameras will be deployed across seven police forces โ Greater Manchester, West Yorkshire, Bedfordshire, Surrey, Sussex, Thames Valley and Hampshire. The Home Office maintains that this technology will only be used to catch'highโharm' offenders with rules to ensure'safeguards and oversight'. According to the government, the technology has already been used to make 580 arrests in London over the last year, including 52 registered sex offenders. However, rights groups have raised concerns that the unprecedented rollout of this surveillance technology risks becoming overly intrusive.
YouTube to Start Using AI to Estimate Users' Ages. Here's What to Know
YouTube is one of the most popular online platforms in the U.S. among all age groups. But not all content on the video-sharing site is appropriate for all ages. While the platform, like most, has restrictions on certain content, such as violence and nudity, for users under 18, these safeguards have in the past been easy for young users to circumvent by entering an older birthdate on their account. But now, the company is rolling out an artificial intelligence-powered tool to estimate a user's age based on their activity on the platform "and then use that signal, regardless of the birthday in the account, to deliver our age-appropriate product experiences and protection," said James Beser, director of product management at YouTube Youth, in blog post last month. The technology, according to Beser, has been used in other markets "for some time" and will begin being tested in the U.S. on Wednesday before a wider rollout.