Law
Rule-based Classifier Models
Di Florio, Cecilia, Dong, Huimin, Rotolo, Antonino
We extend the formal framework of classifier models used in the legal domain. While the existing classifier framework characterises cases solely through the facts involved, legal reasoning fundamentally relies on both facts and rules, particularly the ratio decidendi. This paper presents an initial approach to incorporating sets of rules within a classifier. Our work is built on the work of Canavotto et al. (2023), which has developed the rule-based reason model of precedential constraint within a hierarchy of factors. We demonstrate how decisions for new cases can be inferred using this enriched rule-based classifier framework. Additionally, we provide an example of how the time element and the hierarchy of courts can be used in the new classifier framework.
Extracting Abstraction Dimensions by Identifying Syntax Pattern from Texts
Zhou, Jian, Li, Jiazheng, Zhuge, Sirui, Zhuge, Hai
This paper proposed an approach to automatically discovering subject dimension, action dimension, object dimension and adverbial dimension from texts to efficiently operate texts and support query in natural language. The high quality of trees guarantees that all subjects, actions, objects and adverbials and their subclass relations within texts can be represented. The independency of trees ensures that there is no redundant representation between trees. The expressiveness of trees ensures that the majority of sentences can be accessed from each tree and the rest of sentences can be accessed from at least one tree so that the tree-based search mechanism can support querying in natural language. Experiments show that the average precision, recall and F1-score of the abstraction trees constructed by the subclass relations of subject, action, object and adverbial are all greater than 80%. The application of the proposed approach to supporting query in natural language demonstrates that different types of question patterns for querying subject or object have high coverage of texts, and searching multiple trees on subject, action, object and adverbial according to the question pattern can quickly reduce search space to locate target sentences, which can support precise operation on texts.
Beyond Public Access in LLM Pre-Training Data
Rosenblat, Sruly, O'Reilly, Tim, Strauss, Ilan
Our AU-ROC scores show that GPT-4o, OpenAI's more recent and capable model, demonstrates strong recognition of paywalled O'Reilly book content (AUROC = 82%), compared to OpenAI's earlier model GPT-3.5 Turbo. In contrast, GPT-3.5 Turbo shows greater relative recognition of publicly accessible O'Reilly book samples. GPT-4o Mini, as a much smaller model, shows no knowledge of public or non-public O'Reilly Media content when tested (AUROC 50%). Testing multiple models, with the same cutoff date, helps us account for potential language shifts over time that might bias our findings. These results highlight the urgent need for increased corporate transparency regarding pre-training data sources as a means to develop formal licensing frameworks for AI content training.
TRIED: Truly Innovative and Effective AI Detection Benchmark, developed by WITNESS
Anlen, Shirin, Wojciak, Zuzanna
The proliferation of generative AI and deceptive synthetic media threatens the global information ecosystem, especially across the Global Majority. This report from WITNESS highlights the limitations of current AI detection tools, which often underperform in real-world scenarios due to challenges related to explainability, fairness, accessibility, and contextual relevance. In response, WITNESS introduces the Truly Innovative and Effective AI Detection (TRIED) Benchmark, a new framework for evaluating detection tools based on their real-world impact and capacity for innovation. Drawing on frontline experiences, deceptive AI cases, and global consultations, the report outlines how detection tools must evolve to become truly innovative and relevant by meeting diverse linguistic, cultural, and technological contexts. It offers practical guidance for developers, policy actors, and standards bodies to design accountable, transparent, and user-centered detection solutions, and incorporate sociotechnical considerations into future AI standards, procedures and evaluation frameworks. By adopting the TRIED Benchmark, stakeholders can drive innovation, safeguard public trust, strengthen AI literacy, and contribute to a more resilient global information credibility.
BrowseComp-ZH: Benchmarking Web Browsing Ability of Large Language Models in Chinese
Zhou, Peilin, Leon, Bruce, Ying, Xiang, Zhang, Can, Shao, Yifan, Ye, Qichen, Chong, Dading, Jin, Zhiling, Xie, Chenxuan, Cao, Meng, Gu, Yuxin, Hong, Sixin, Ren, Jing, Chen, Jian, Liu, Chao, Hua, Yining
As large language models (LLMs) evolve into tool-using agents, the ability to browse the web in real-time has become a critical yardstick for measuring their reasoning and retrieval competence. Existing benchmarks such as BrowseComp concentrate on English and overlook the linguistic, infrastructural, and censorship-related complexities of other major information ecosystems -- most notably Chinese. To address this gap, we introduce BrowseComp-ZH, a high-difficulty benchmark purpose-built to comprehensively evaluate LLM agents on the Chinese web. BrowseComp-ZH consists of 289 multi-hop questions spanning 11 diverse domains. Each question is reverse-engineered from a short, objective, and easily verifiable answer (e.g., a date, number, or proper noun). A two-stage quality control protocol is applied to strive for high question difficulty and answer uniqueness. We benchmark over 20 state-of-the-art language models and agentic search systems on our proposed BrowseComp-ZH. Despite their strong conversational and retrieval capabilities, most models struggle severely: a large number achieve accuracy rates below 10%, and only a handful exceed 20%. Even the best-performing system, OpenAI's DeepResearch, reaches just 42.9%. These results demonstrate the considerable difficulty of BrowseComp-ZH, where success demands not only effective retrieval strategies, but also sophisticated reasoning and information reconciliation -- capabilities that current models still struggle to master. Our dataset, construction guidelines, and benchmark results have been publicly released at https://github.com/PALIN2018/BrowseComp-ZH.
A Judge Says Meta's AI Copyright Case Is About 'the Next Taylor Swift'
US District Court Judge Vince Chhabria spent several hours grilling lawyers from both sides after they each filed motions for partial summary judgment, meaning they want Chhabria to rule on specific issues of the case rather than leaving each one to be decided at trial. The authors allege that Meta illegally used their work to build its generative AI tools, emphasizing that the company pirated their books through "shadow libraries" like LibGen. Kadrey v. Meta is one of the dozens of lawsuits filed against AI companies that are winding through the US legal system. While the authors were heavily focused on the piracy element of the case, Chhabria spoke emphatically about his belief that the big question is whether Meta's AI tools will hurt book sales and otherwise cause the authors to lose money. "If you are dramatically changing, you might even say obliterating, the market for that person's work, and you're saying that you don't even have to pay a license to that person to use their work to create the product that's destroying the market for their work--I just don't understand how that can be fair use," he told Meta lawyer Kannon Shanmugam.
Cloobeck sues Villaraigosa over use of the phrase 'proven problem solver'
In an unusual twist in the governor's race, a wealthy Democratic businessman is suing former Los Angeles Mayor Antonio Villaraigosa over the use of a common phrase in political campaigns. Stephen Cloobeck, a philanthropist and Democratic donor who made his fortune in real estate and hospitality, filed a lawsuit against Villaraigosa this week after the former mayor repeatedly described himself as a "proven problem solver" in campaign materials. Cloobeck, who has applied for a federal trademark of the phrase "I am a proven problem solver," texted the federal lawsuit to Villaraigosa late Tuesday, though the former mayor has not been served yet. The lawsuit argues that Cloobeck has been using the phrase since March 2024, and that "it has acquired extensive goodwill, developed a high degree of distinctiveness, and become famous, well known, and recognized as identifying Cloobeck's campaign." "In light of the fame, acquired goodwill, and overall consumer recognition of [the phrase Cloobeck is seeking to patent, he] is very concerned that the public will likely be confused or mistakenly believe that Villaraigosa's campaign is endorsed, approved, sponsored by, or affiliated, connected, or associated with" Villaraigosa, the suit alleges.
Public Opinion and The Rise of Digital Minds: Perceived Risk, Trust, and Regulation Support
Bullock, Justin B., Pauketat, Janet V. T., Huang, Hsini, Wang, Yi-Fan, Anthis, Jacy Reese
Governance institutions must respond to societal risks, including those posed by generative AI. This study empirically examines how public trust in institutions and AI technologies, along with perceived risks, shape preferences for AI regulation. Using the nationally representative 2023 Artificial Intelligence, Morality, and Sentience (AIMS) survey, we assess trust in government, AI companies, and AI technologies, as well as public support for regulatory measures such as slowing AI development or outright bans on advanced AI. Our findings reveal broad public support for AI regulation, with risk perception playing a significant role in shaping policy preferences. Individuals with higher trust in government favor regulation, while those with greater trust in AI companies and AI technologies are less inclined to support restrictions. Trust in government and perceived risks significantly predict preferences for both soft (e.g., slowing development) and strong (e.g., banning AI systems) regulatory interventions. These results highlight the importance of public opinion in AI governance. As AI capabilities advance, effective regulation will require balancing public concerns about risks with trust in institutions. This study provides a foundational empirical baseline for policymakers navigating AI governance and underscores the need for further research into public trust, risk perception, and regulatory strategies in the evolving AI landscape.
Characterizing AI Agents for Alignment and Governance
Kasirzadeh, Atoosa, Gabriel, Iason
The creation of effective governance mechanisms for AI agents requires a deeper understanding of their core properties and how these properties relate to questions surrounding the deployment and operation of agents in the world. This paper provides a characterization of AI agents that focuses on four dimensions: autonomy, efficacy, goal complexity, and generality. We propose different gradations for each dimension, and argue that each dimension raises unique questions about the design, operation, and governance of these systems. Moreover, we draw upon this framework to construct "agentic profiles" for different kinds of AI agents. These profiles help to illuminate cross-cutting technical and non-technical governance challenges posed by different classes of AI agents, ranging from narrow task-specific assistants to highly autonomous general-purpose systems. By mapping out key axes of variation and continuity, this framework provides developers, policymakers, and members of the public with the opportunity to develop governance approaches that better align with collective societal goals.
Who Gets the Callback? Generative AI and Gender Bias
Chaturvedi, Sugat, Chaturvedi, Rochana
Generative artificial intelligence (AI), particularly large language models (LLMs), is being rapidly deployed in recruitment and for candidate shortlisting. We audit several mid-sized open-source LLMs for gender bias using a dataset of 332,044 real-world online job postings. For each posting, we prompt the model to recommend whether an equally qualified male or female candidate should receive an interview callback. We find that most models tend to favor men, especially for higher-wage roles. Mapping job descriptions to the Standard Occupational Classification system, we find lower callback rates for women in male-dominated occupations and higher rates in female-associated ones, indicating occupational segregation. A comprehensive analysis of linguistic features in job ads reveals strong alignment of model recommendations with traditional gender stereotypes. To examine the role of recruiter identity, we steer model behavior by infusing Big Five personality traits and simulating the perspectives of historical figures. We find that less agreeable personas reduce stereotyping, consistent with an agreeableness bias in LLMs. Our findings highlight how AI-driven hiring may perpetuate biases in the labor market and have implications for fairness and diversity within firms.