The rapid scaling of AI has spurred a growing emphasis on ethical considerations in both development and practice. This has led to the formulation of increasingly sophisticated model auditing and reporting requirements, as well as governance frameworks to mitigate potential risks to individuals and society. At this critical juncture, we review the practical challenges of promoting responsible AI and transparency in informal sectors like OSS that support vital infrastructure and see widespread use. We focus on how model performance evaluation may inform or inhibit probing of model limitations, biases, and other risks. Our controlled analysis of 7903 Hugging Face projects found that risk documentation is strongly associated with evaluation practices. Yet, submissions (N=789) from the platform's most popular competitive leaderboard showed less accountability among high performers. Our findings can inform AI providers and legal scholars in designing interventions and policies that preserve open-source innovation while incentivizing ethical uptake.

English is the international standard of social research, but scholars are increasingly conscious of their responsibility to meet the need for scholarly insight into communication processes globally. This tension is as true in computational methods as any other area, with revolutionary advances in the tools for English language texts leaving most other languages far behind. In this paper, we aim to leverage those very advances to demonstrate that multi-language analysis is currently accessible to all computational scholars. We show that English-trained measures computed after translation to English have adequate-to-excellent accuracy compared to source-language measures computed on original texts. We show this for three major analytics -- sentiment analysis, topic analysis, and word embeddings -- over 16 languages, including Spanish, Chinese, Hindi, and Arabic. We validate this claim by comparing predictions on original language tweets and their backtranslations: double translations from their source language to English and back to the source language. Overall, our results suggest that Google Translate, a simple and widely accessible tool, is effective in preserving semantic content across languages and methods. Modern machine translation can thus help computational scholars make more inclusive and general claims about human communication.

Every day, millions of people gather on online game servers to collaborate in real time toward shared goals. What may seem like frivolous activity may, when investigated more closely, provide revolutionary opportunities to advance the science of teamwork. Teamwork is an important part of modern society, however, collaboration between people is often made difficult due to differing ideals, opinions, and personality types. We leverage a popular self-hosted multiplayer online game environment to design a framework for developing and deploying tasks that elicit different kinds of teamwork. We propose to use these to capture fine-scale details of individual and group performance across environments. The game in which we implement this system, Minecraft, is ideal because it is heavily modifiable and already enjoys a vast user base of surprising gender, age, and ethnic diversity. We heavily modify the game in order to introduce new mechanics that facilitate collaboration, monitor activity, and manipulate group composition, all to provide the groundwork for deeper quantitative insights into effective teams.

We present Heapcraft: an open-source suite of tools for monitoring and improving collaboration in Minecraft. At the core of our system is a data collection and analysis framework for recording gameplay. We collected over 3451 player-hours of game behavior from 908 different players, and performed a general study of online collaboration. To make our game analytics easily accessible, we developed interactive information visualization tools and an analysis framework for players, administrators, and researchers to explore graphs, maps and timelines of live server activity. As part of our research, we introduce the collaboration index, a metric which allows server administrators and researchers to quantify, predict, and improve collaboration on Minecraft servers. Our analysis reveals several possible predictors of collaboration which can be used to improve collaboration on Minecraft servers. Heapcraft is designed to be general, and has the potential to be used for other shared online virtual worlds.