Foundation model development attracts a rapidly expanding body of contributors, scientists, and applications. To help shape responsible development practices, we introduce the Foundation Model Development Cheatsheet: a growing collection of 250+ tools and resources spanning text, vision, and speech modalities. We draw on a large body of prior work to survey resources (e.g. software, documentation, frameworks, guides, and practical tools) that support informed data selection, processing, and understanding, precise and limitation-aware artifact documentation, efficient model training, advance awareness of the environmental impact from training, careful model evaluation of capabilities, risks, and claims, as well as responsible model release, licensing and deployment practices. We hope this curated collection of resources helps guide more responsible development. The process of curating this list, enabled us to review the AI development ecosystem, revealing what tools are critically missing, misused, or over-used in existing practices. We find that (i) tools for data sourcing, model evaluation, and monitoring are critically under-serving ethical and real-world needs, (ii) evaluations for model safety, capabilities, and environmental impact all lack reproducibility and transparency, (iii) text and particularly English-centric analyses continue to dominate over multilingual and multi-modal analyses, and (iv) evaluation of systems, rather than just models, is needed so that capabilities and impact are assessed in context.

The widespread application of Large Language Models (LLMs) across various tasks and fields has necessitated the alignment of these models with human values and preferences. Given various approaches of human value alignment, ranging from Reinforcement Learning with Human Feedback (RLHF), to constitutional learning, etc. there is an urgent need to understand the scope and nature of human values injected into these models before their release. There is also a need for model alignment without a costly large scale human annotation effort. We propose UniVaR, a high-dimensional representation of human value distributions in LLMs, orthogonal to model architecture and training data. Trained from the value-relevant output of eight multilingual LLMs and tested on the output from four multilingual LLMs, namely LlaMA2, ChatGPT, JAIS and Yi, we show that UniVaR is a powerful tool to compare the distribution of human values embedded in different LLMs with different langauge sources. Through UniVaR, we explore how different LLMs prioritize various values in different languages and cultures, shedding light on the complex interplay between human values and language modeling.

In this work, we introduce a comprehensive error typology specifically designed for evaluating two distinct tasks in machine-generated patent texts: claims-to-abstract generation, and the generation of the next claim given previous ones. We have also developed a benchmark, PatentEval, for systematically assessing language models in this context. Our study includes a comparative analysis, annotated by humans, of various models. These range from those specifically adapted during training for tasks within the patent domain to the latest general-purpose large language models (LLMs). Furthermore, we explored and evaluated some metrics to approximate human judgments in patent text evaluation, analyzing the extent to which these metrics align with expert assessments. These approaches provide valuable insights into the capabilities and limitations of current language models in the specialized field of patent text generation.

In his classic essay, "The Normative Structure of Science," sociologist Robert K. Merton identified universalism as a foundational principle of the scientific enterprise, one that distinguishes science from other competing systems of knowing. According to Merton and Storer's formulation (Merton and Storer, 1973, p. 270), universalism holds that the evaluation of scientific contributions "is not to depend on the personal or social attributes of their protagonist; his race, nationality, religion, class, and personal qualities are as such irrelevant." The value of universalism is manifested perhaps most concretely in the practice of double-blind peer review, wherein the identities of both those making scientific claims and those evaluating them are obscured from one another (Bornmann, 2011). While scholars have long observed that adherence to the principle of universalism is far from universal (Mulkay, 1976; Cole, 1992; Long and Fox, 1995), the growing availability of large-scale databases is creating opportunities for unprecedented insight into processes of scientific evaluation (Teplitskiy et al., 2018; Dondio et al., 2019; Lane et al., 2021), including the barriers that inhibit objective assessments. Recent literature in particular has raised considerable concern about the role of gender in scientific evaluation (Moss-Racusin et al., 2012; Reuben et al., 2014; Oliveira et al., 2019; Card et al., 2020a).

A scientist has unveiled a concept for a prison of the future that he has claimed would fast-track a criminal's release to minutes, instead of years or decades. Called Cognify, the design would implant synthetic memories of a person's crime into their brain, but showing their victim's perspective. The system could feature a VR-like device that displays AI-generated footage of the offence, coupled with a brain implant that induces emotional states like remorse or regret - feelings some individuals may not produce on their own. The concept, developed by Hashem Al-Ghaili, would ensure the long-term effects of the therapy session by making the memories permanent. Called Cognify, the design would implant synthetic memories of a person's crime into their brain, but showing their victim's perspective There are more than 1.7 million people currently incarcerated in the US.

Supporters have compared machine learning by AI tools to the way humans learn by reading, hearing and seeing previous works. But in the complaints, which were filed in federal court in Massachusetts and New York, the record labels say the AI firms are simply making money from having copied the songs. The complaints say Suno and Udio produce works like "Prancing Queen" that even devoted ABBA fans would struggle to distinguish from an authentic recording from the band. Songs cited in the Udio lawsuit include Mariah Carey's "All I Want for Christmas is You" and "My Girl" by The Temptations. They said there was nothing about AI that excused the firms from "playing by the rules" and warned that the "wholesale theft" of the recordings threatened "the entire music ecosystem". The lawsuits come just months after roughly 200 artists including Billie Eilish and Nicki Minaj signed a letter calling for the "predatory" use of artificial intelligence (AI) in the music industry to be stopped.

Major music labels are taking on AI startups that they believe trained on their songs without paying. The filings against the AI companies reportedly demand injunctions against future use and damages of up to 150,000 per infringed work. The suits appear aimed at establishing licensed training as the only acceptable industry framework for AI moving forward -- while instilling fear in companies that train their models without consent. Suno AI and Udio AI (Uncharted Labs run the latter) are startups with software that generates music based on text inputs. The former is a partner of Microsoft for its CoPilot music generation tool.

The music industry has officially declared war on Suno and Udio, two of the most prominent AI music generators. The plaintiffs seek damages up to 150,000 per work infringed. The lawsuit against Suno is filed in Massachusetts, while the case against Udio's parent company Uncharted Inc. was filed in New York. Suno and Udio did not immediately respond to a request to comment. "Unlicensed services like Suno and Udio that claim it's'fair' to copy an artist's life's work and exploit it for their own profit without consent or pay set back the promise of genuinely innovative AI for us all," Recording Industry Association of America chairman and CEO Mitch Glazier said in a press release.

Geologists have raised concerns about potential Chinese censorship and bias in a chatbot being developed with the backing of the International Union of Geological Sciences (IUGS), one of the world's largest scientific organisations and a Unesco partner. The GeoGPT chatbot is aimed at geoscientists and researchers, particularly in the global south, to help them develop their understanding of earth sciences by drawing on swaths of data and research on billions of years of the planet's history. It is an initiative from Deep-time Digital Earth (DDE), a largely Chinese-funded programme founded in 2019 to enhance international scientific cooperation and help countries to realise the UN's sustainable development goals. Part of the underlying AI for GeoGPT is Qwen, a large language model built by the Chinese tech company Alibaba. Responding to the article, DDE representatives Michael Stephenson, Hans Thybo, Chengshan Wang and Ishwaran Natarajan said the chatbot also used Meta's Llama, another large language model, and that during testing they had not noticed any state censorship, which they said was "unlikely" given that the system was "based entirely in geoscience information".

As large language models (LLMs) become increasingly prevalent in a wide variety of applications, concerns about the safety of their outputs have become more significant. Most efforts at safety-tuning or moderation today take on a predominantly Western-centric view of safety, especially for toxic, hateful, or violent speech. In this paper, we describe LionGuard, a Singapore-contextualized moderation classifier that can serve as guardrails against unsafe LLM outputs. When assessed on Singlish data, LionGuard outperforms existing widely-used moderation APIs, which are not finetuned for the Singapore context, by 14% (binary) and up to 51% (multi-label). Our work highlights the benefits of localization for moderation classifiers and presents a practical and scalable approach for low-resource languages.