Curriculum
Interview with Gillian Hadfield: Normative infrastructure for AI alignment
During the 33rd International Joint Conference on Artificial Intelligence (IJCAI), held in Jeju, I had the opportunity to meet with one of the keynote speakers, Gillian Hadfield. We spoke about her interdisciplinary research, career trajectory, path into AI alignment, law, and general thoughts on AI systems. Transcript: Note: the transcript has been lightly edited for clarity. This is an interview with Professor Gillian Hadfield who was a keynote speaker at IJCAI 2024. She gave a very insightful talk about normative infrastructures and how they can guide our search for AI alignment. Kumar Kshitij Patel (KKP): Could you talk a bit about your background and career trajectory? I want our readers to understand how much interdisciplinary work you've done over the years. Gillian Hadfield (GH): I did a PhD in economics and a law degree, a JD, at Stanford, originally motivated by wanting to think about the big questions about the world. So I read John Rawls' theory of justice when I was an undergraduate, and those are the big questions: how do we organize the world and just institutions, but I was very interested in using more formal methods and social scientific approaches. That's why I decided to do that joint degree. So, this is in the 1980s, and in the early days of starting to use a lot of game theory. I studied information theory, a student of Canaro and Paul Milgram at the economics department at Stanford. I did work on contract theory, bargaining theory, but I was still very interested in going to law school, not to practice law, but to learn about legal institutions and how those work. I was a member of this emerging area of law and economics early in my career, which of course, was interdisciplinary, using economics to think about law and legal institutions.
Google and Duolingo think AI can change the way we learn languages. Are they right?
AI continues to expand its reach into our lives, and language learning is next on the list. This week brought big developments from both Google and Duolingo on that front. On Google's end, the search giant launched new Gemini-powered AI tools for users to learn foreign languages. Dubbed Little Language Lessons, the experimental feature offers three interactive lessons that "personalize language learning." For instance, "Tiny Lesson" can help you learn phrases for specific situations (such as losing your passport), while "Slang Hang" helps users learn local slang for less-stuffy conversation.
Grab an exclusive price on a Babbel lifetime subscription
TL;DR: It's only 129.99 to get a Babbel Language Learning Lifetime Subscription (all languages) with code LEARN40, but this sale ends soon. If you had all the time in the world, what language would you learn? With a lifetime subscription to Babbel, you can take your time learning any of 14 different languages, and it's only 129.99 (reg. The beauty of a lifetime subscription is that it never expires, meaning you can study Spanish for your next trip, brush up on German for business, or finally learn Italian to impress your friends, all at your own pace. With Babbel's structured lessons, you'll find that learning new languages can be a continuous journey of personal growth and intellectual stimulation.
Babbel might just help you impress the locals by teaching you how to order gelato in Italian
So, you've decided to become the person who can order the perfect cappuccino or savory meal in Italy without fumbling over the word "grazie" like a tourist. Good news: Babbel has you covered for life--and we're talking about 14 languages, not just one. This isn't your high school Spanish class where you learned how to ask where the library is. With Babbel, you'll be swapping stories with locals or negotiating business deals in multiple languages before you know it. Get lifetime access to 14 languages on sale for 129.99 with code LEARN40 through May 4. Here's the best part: learning a new language doesn't just make you look smart.
State Bar of California admits it used AI to develop exam questions, triggering new furor
Nearly two months after hundreds of prospective California lawyers complained that their bar exams were plagued with technical problems and irregularities, the state's legal licensing body has caused fresh outrage by admitting that some multiple-choice questions were developed with the aid of artificial intelligence. The State Bar of California said in a news release Monday that it will ask the California Supreme Court to adjust test scores for those who took its February bar exam. But it declined to acknowledge significant problems with its multiple-choice questions -- even as it revealed that a subset of questions were recycled from a first-year law student exam, while others were developed with the assistance of AI by ACS Ventures, the State Bar's independent psychometrician. "The debacle that was the February 2025 bar exam is worse than we imagined," said Mary Basick, assistant dean of academic skills at UC Irvine Law School. Having the questions drafted by non-lawyers using ...
'I sent AI to art school!' The postmodern master who taught a machine to beef up his old work
By the time you read this article, there's a good chance it will have already been scanned by an artificially intelligent machine. If asked about the artist David Salle, large language models such as ChatGPT or Gemini may repurpose some of the words below to come up with their answer. The bigger the data set, the more convincing the response – and Salle has been written about exhaustively since he first rose to art world stardom in the 1980s. The question is whether AI can ever say anything new about the artist and his work, or if it's for ever condemned to generate more of the same. A similar question lingers beneath the surface of the paintings that Salle has been making since 2023, a new series of which he has just unveiled at Thaddaeus Ropac in London.
The Fragility of Fairness: Causal Sensitivity Analysis for Fair Machine Learning Department of Statistics Department of Statistics University of Oxford
Fairness metrics are a core tool in the fair machine learning literature (FairML), used to determine that ML models are, in some sense, "fair." Real-world data, however, are typically plagued by various measurement biases and other violated assumptions, which can render fairness assessments meaningless. We adapt tools from causal sensitivity analysis to the FairML context, providing a general framework which (1) accommodates effectively any combination of fairness metric and bias that can be posed in the "oblivious setting"; (2) allows researchers to investigate combinations of biases, resulting in non-linear sensitivity; and (3) enables flexible encoding of domain-specific constraints and assumptions. Employing this framework, we analyze the sensitivity of the most common parity metrics under 3 varieties of classifier across 14 canonical fairness datasets. Our analysis reveals the striking fragility of fairness assessments to even minor dataset biases. We show that causal sensitivity analysis provides a powerful and necessary toolkit for gauging the informativeness of parity metric evaluations. Our repository is available here.
Improving Environment Novelty Quantification for Effective Unsupervised Environment Design
Unsupervised Environment Design (UED) formalizes the problem of autocurricula through interactive training between a teacher agent and a student agent. The teacher generates new training environments with high learning potential, curating an adaptive curriculum that strengthens the student's ability to handle unseen scenarios. Existing UED methods mainly rely on regret, a metric that measures the difference between the agent's optimal and actual performance, to guide curriculum design. Regret-driven methods generate curricula that progressively increase environment complexity for the student but overlook environment novelty-a critical element for enhancing an agent's generalizability. Measuring environment novelty is especially challenging due to the underspecified nature of environment parameters in UED, and existing approaches face significant limitations. To address this, this paper introduces the Coverage-based Evaluation of Novelty In Environment (CENIE) framework. CENIE proposes a scalable, domainagnostic, and curriculum-aware approach to quantifying environment novelty by leveraging the student's state-action space coverage from previous curriculum experiences. We then propose an implementation of CENIE that models this coverage and measures environment novelty using Gaussian Mixture Models.
A Synthetic Dataset for Personal Attribute Inference Hanna Yukhymenko
Recently powerful Large Language Models (LLMs) have become easily accessible to hundreds of millions of users world-wide. However, their strong capabilities and vast world knowledge do not come without associated privacy risks. In this work, we focus on the emerging privacy threat LLMs pose - the ability to accurately infer personal information from online texts. Despite the growing importance of LLM-based author profiling, research in this area has been hampered by a lack of suitable public datasets, largely due to ethical and privacy concerns associated with real personal data. We take two steps to address this problem: (i) we construct a simulation framework for the popular social media platform Reddit using LLM agents seeded with synthetic personal profiles; (ii) using this framework, we generate SynthPAI, a diverse synthetic dataset of over 7800 comments manually labeled for personal attributes. We validate our dataset with a human study showing that humans barely outperform random guessing on the task of distinguishing our synthetic comments from real ones. Further, we verify that our dataset enables meaningful personal attribute inference research by showing across 18 state-of-theart LLMs that our synthetic comments allow us to draw the same conclusions as real-world data. Combined, our experimental results, dataset and pipeline form a strong basis for future privacy-preserving research geared towards understanding and mitigating inference-based privacy threats that LLMs pose.