Goto

Collaborating Authors

 Law


Salamandra Technical Report

arXiv.org Artificial Intelligence

This work introduces Salamandra, a suite of open-source decoder-only large language models available in three different sizes: 2, 7, and 40 billion parameters. The models were trained from scratch on highly multilingual data that comprises text in 35 European languages and code. Our carefully curated corpus is made exclusively from open-access data compiled from a wide variety of sources. Along with the base models, supplementary checkpoints that were fine-tuned on public-domain instruction data are also released for chat applications. Additionally, we also share our preliminary experiments on multimodality, which serve as proof-of-concept to showcase potential applications for the Salamandra family. Our extensive evaluations on multilingual benchmarks reveal that Salamandra has strong capabilities, achieving competitive performance when compared to similarly sized open-source models. We provide comprehensive evaluation results both on standard downstream tasks as well as key aspects related to bias and safety.With this technical report, we intend to promote open science by sharing all the details behind our design choices, data curation strategy and evaluation methodology. In addition to that, we deviate from the usual practice by making our training and evaluation scripts publicly accessible. We release all models under a permissive Apache 2.0 license in order to foster future research and facilitate commercial use, thereby contributing to the open-source ecosystem of large language models.


Faithful, Unfaithful or Ambiguous? Multi-Agent Debate with Initial Stance for Summary Evaluation

arXiv.org Artificial Intelligence

Faithfulness evaluators based on large language models (LLMs) are often fooled by the fluency of the text and struggle with identifying errors in the summaries. We propose an approach to summary faithfulness evaluation in which multiple LLM-based agents are assigned initial stances (regardless of what their belief might be) and forced to come up with a reason to justify the imposed belief, thus engaging in a multi-round debate to reach an agreement. The uniformly distributed initial assignments result in a greater diversity of stances leading to more meaningful debates and ultimately more errors identified. Furthermore, by analyzing the recent faithfulness evaluation datasets, we observe that naturally, it is not always the case for a summary to be either faithful to the source document or not. We therefore introduce a new dimension, ambiguity, and a detailed taxonomy to identify such special cases. Experiments demonstrate our approach can help identify ambiguities, and have even a stronger performance on non-ambiguous summaries.


FLAME: Flexible LLM-Assisted Moderation Engine

arXiv.org Artificial Intelligence

The rapid advancement of Large Language Models (LLMs) has introduced significant challenges in moderating user-model interactions. While LLMs demonstrate remarkable capabilities, they remain vulnerable to adversarial attacks, particularly ``jailbreaking'' techniques that bypass content safety measures. Current content moderation systems, which primarily rely on input prompt filtering, have proven insufficient, with techniques like Best-of-N (BoN) jailbreaking achieving success rates of 80% or more against popular LLMs. In this paper, we introduce Flexible LLM-Assisted Moderation Engine (FLAME): a new approach that shifts the focus from input filtering to output moderation. Unlike traditional circuit-breaking methods that analyze user queries, FLAME evaluates model responses, offering several key advantages: (1) computational efficiency in both training and inference, (2) enhanced resistance to BoN jailbreaking attacks, and (3) flexibility in defining and updating safety criteria through customizable topic filtering. Our experiments demonstrate that FLAME significantly outperforms current moderation systems. For example, FLAME reduces attack success rate in GPT-4o-mini and DeepSeek-v3 by a factor of ~9, while maintaining low computational overhead. We provide comprehensive evaluation on various LLMs and analyze the engine's efficiency against the state-of-the-art jailbreaking. This work contributes to the development of more robust and adaptable content moderation systems for LLMs.


Logical Lease Litigation: Prolog and LLMs for Rental Law Compliance in New York

arXiv.org Artificial Intelligence

Legal cases require careful logical reasoning following the laws, whereas interactions with non- technical users must be in natural language. As an application combining logical reasoning using Prolog and natural language processing using large language models (LLMs), this paper presents a novel approach and system, LogicLease, to automate the analysis of landlord-tenant legal cases in the state of New York. LogicLease determines compliance with relevant legal requirements by analyzing case descriptions and citing all relevant laws. It leverages LLMs for information extraction and Prolog for legal reasoning. By separating information extraction from legal reasoning, LogicLease achieves greater transparency and control over the legal logic applied to each case. We evaluate the accuracy, efficiency, and robustness of LogicLease through a series of tests, achieving 100% accuracy and an average processing time of 2.57 seconds. LogicLease presents advantages over state-of-the-art LLM- based legal analysis systems by providing clear, step-by-step reasoning, citing specific laws, and distinguishing itself by its ability to avoid hallucinations - a common issue in LLMs.


Censor Dependent Variational Inference

arXiv.org Machine Learning

This paper provides a comprehensive analysis of variational inference in latent variable models for survival analysis, emphasizing the distinctive challenges associated with applying variational methods to survival data. We identify a critical weakness in the existing methodology, demonstrating how a poorly designed variational distribution may hinder the objective of survival analysis tasks--modeling time-to-event distributions. We prove that the optimal variational distribution, which perfectly bounds the log-likelihood, may depend on the censoring mechanism. To address this issue, we propose censor-dependent variational inference (CDVI), tailored for latent variable models in survival analysis. More practically, we introduce CD-CVAE, a V-structure Variational Autoencoder (VAE) designed for the scalable implementation of CDVI. Further discussion extends some existing theories and training techniques to survival analysis. Extensive experiments validate our analysis and demonstrate significant improvements in the estimation of individual survival distributions.


Digital Access Is Critical for Society Say Industry Leaders

TIME - Tech

Improving connectivity can both benefit those who most need it most and boost the businesses that provide the service. That's the case telecom industry leaders made during a panel on Feb. 11 at the World Governments Summit in Dubai. Titled "Can we innovate our way to a more connected world?", the panel was hosted by TIME's Editor-in-Chief Sam Jacobs. During the course of the conversation, Margherita Della Valle, CEO of U.K.-based multinational telecom company Vodafone Group, said, "For society today, connectivity is essential. We are moving from the old divide in the world between the haves and the have-nots towards a new divide, which is between those who have access to connectivity and those who don't."


Google defends scrapping AI pledges and DEI goals in all-staff meeting

The Guardian

Google's executives gave details on Wednesday on how the tech giant will sunset its diversity initiatives and defended dropping its pledge against building artificial intelligence for weaponry and surveillance in an all-staff meeting. Melonie Parker, Google's former head of diversity, said the company was doing away with its diversity and inclusion employee training programs and "updating" broader training programs that have "DEI content". It was the first time company executives have addressed the whole staff since Google announced it would no longer follow hiring goals for diversity and took down its pledge not to build militarized AI. The chief legal officer, Kent Walker, said a lot had changed since Google first introduced its AI principles in 2018, which explicitly stated Google would not build AI for harmful purposes. He said it would be "good for society" for the company to be part of evolving geopolitical discussions in response to a question about why the company removed prohibitions against building AI for weapons and surveillance.


Paris AI summit: Why won't US, UK sign global artificial intelligence pact?

Al Jazeera

The United States and United Kingdom have refused to sign an Artificial Intelligence Action Summit declaration calling for policies "ensuring AI is open, inclusive, transparent, ethical, safe, secure and trustworthy". The summit in Paris on Monday and Tuesday brought together representatives from more than 100 countries to discuss how to reach a consensus on guiding the development of AI. "We are still in the early days, and I already believe AI will be the most profound shift of our lifetimes," Google CEO Sundar Pichai told the summit. The meeting, which was held amid a three-way race for AI dominance, revealed a divide in the priorities of some nations. While Europe is seeking to regulate and invest, China is focused on expanding access through state-backed tech giants, and the US is pushing for a hands-off approach in terms of regulation. Here's what you need to know about the summit and the AI race: Some leaders at the summit emphasised the need for the creation of a diverse and inclusive AI "ecosystem" that is human rights-based, ethical, safe and trustworthy.


Elon Musk owning OpenAI would be a terrible idea. That doesn't mean it won't happen Chris Stokel-Walker

The Guardian

The two had a blowout argument over the future direction of OpenAI – the company they came together to found in 2015 – with Altman seemingly content to pursue a for-profit approach and Musk feeling that was forswearing the founding principles of the firm as well as its name. OpenAI couldn't be open, he reckoned, if it was closed off and trying to make money rather than better humanity. So it's no surprise that Musk, who lodged an audacious bid to take over Twitter a little more than two years ago, which ended up with his ownership of the platform now called X, has sought to put a spoiler in two years of near-untrammelled growth for OpenAI. Musk – who is currently overhauling (to his supporters; "tearing down" to his opponents) the US government to be, as he would describe it, leaner and more efficient while also devastating important programmes such as international aid and cutting-edge scientific research – has lodged a near 100bn bid for OpenAI's non-profit arm. "It's time for OpenAI to return to the open-source, safety-focused force for good it once was," Musk said in a statement supplied by the lawyer shepherding his bid.


AgentSociety: Large-Scale Simulation of LLM-Driven Generative Agents Advances Understanding of Human Behaviors and Society

arXiv.org Artificial Intelligence

Understanding human behavior and society is a central focus in social sciences, with the rise of generative social science marking a significant paradigmatic shift. By leveraging bottom-up simulations, it replaces costly and logistically challenging traditional experiments with scalable, replicable, and systematic computational approaches for studying complex social dynamics. Recent advances in large language models (LLMs) have further transformed this research paradigm, enabling the creation of human-like generative social agents and realistic simulacra of society. In this paper, we propose AgentSociety, a large-scale social simulator that integrates LLM-driven agents, a realistic societal environment, and a powerful large-scale simulation engine. Based on the proposed simulator, we generate social lives for over 10k agents, simulating their 5 million interactions both among agents and between agents and their environment. Furthermore, we explore the potential of AgentSociety as a testbed for computational social experiments, focusing on four key social issues: polarization, the spread of inflammatory messages, the effects of universal basic income policies, and the impact of external shocks such as hurricanes. These four issues serve as valuable cases for assessing AgentSociety's support for typical research methods -- such as surveys, interviews, and interventions -- as well as for investigating the patterns, causes, and underlying mechanisms of social issues. The alignment between AgentSociety's outcomes and real-world experimental results not only demonstrates its ability to capture human behaviors and their underlying mechanisms, but also underscores its potential as an important platform for social scientists and policymakers.