Breakthroughs, discoveries, and DIY tips sent every weekday. Humans don't need to blast off into space to find some truly alien-looking wonders. The deepest depths of our ocean are like another planet, teeming with the charismatic "Casper" octopus, the carnivorous (aka the flying spaghetti monster), and even some sharks . A team from Kumamoto University in Japan recently uncovered a deep-sea anemone that has a tight bond with hermit crabs. These wispy pink invertebrates build shell-like "homes" for the crabs.

The alignment of large language models (LLMs) with human values increasingly relies on using other LLMs as automated judges, or ``autoraters''. However, their reliability is limited by a foundational issue: they are trained on discrete preference labels, forcing a single ground truth onto tasks that are often subjective, ambiguous, or nuanced. We argue that a reliable autorater must learn to model the full distribution of preferences defined by a target population. In this paper, we propose a general framework for calibrating probabilistic autoraters to any given preference distribution. We formalize the problem and present two learning methods tailored to different data conditions: 1) a direct supervised fine-tuning for dense, probabilistic labels, and 2) a reinforcement learning approach for sparse, binary labels. Our empirical results show that finetuning autoraters with a distribution-matching objective leads to verbalized probability predictions that are better aligned with the target preference distribution, with improved calibration and significantly lower positional bias, all while preserving performance on objective tasks.

At RE•WORK, we are strong advocates for supporting women working towards advancing technology, so ahead of the upcoming AI in Healthcare Summit, we set out to highlight inspirational women within the US healthcare and pharma sectors who are working at the forefront of AI developments, and who deserve recognition for their achievements. While we set out to create a list of just 20 – we couldn't narrow it down, as there are so many inspiring and prominent females in this space! Hear from many of them at our AI in Healthcare Summit, and more outside the healthcare space at our Women in AI Reception, both being held in Boston next month. Help us to continue highlighting leading women in AI by nominating your influential woman for our next edition. RE•WORK holds Women in AI events, podcasts, and blogs.

FREIGHTER GAP, Wyo.--On blisteringly hot desert sands, researchers crawled on their hands and knees avoiding fist-size cacti littering the ground. Their goal: collecting bones and teeth of some of the earliest known primates to shed light on the adaptations at the root of the evolutionary lineage that led to humans. The fossils, though, are the size of a fingernail or smaller, and they are scattered over an area of about 10,000 square kilometers in the rocky desert of Wyoming's Great Divide Basin. That's a lot of ground to cover, especially on all fours and in searing heat. So the scientists are relying on a tool never tried before in paleontology: artificial intelligence.