Goto

Collaborating Authors

 kill bill


Learning Correlated Reward Models: Statistical Barriers and Opportunities

arXiv.org Machine Learning

Random Utility Models (RUMs) are a classical framework for modeling user preferences and play a key role in reward modeling for Reinforcement Learning from Human Feedback (RLHF). However, a crucial shortcoming of many of these techniques is the Independence of Irrelevant Alternatives (IIA) assumption, which collapses \emph{all} human preferences to a universal underlying utility function, yielding a coarse approximation of the range of human preferences. On the other hand, statistical and computational guarantees for models avoiding this assumption are scarce. In this paper, we investigate the statistical and computational challenges of learning a \emph{correlated} probit model, a fundamental RUM that avoids the IIA assumption. First, we establish that the classical data collection paradigm of pairwise preference data is \emph{fundamentally insufficient} to learn correlational information, explaining the lack of statistical and computational guarantees in this setting. Next, we demonstrate that \emph{best-of-three} preference data provably overcomes these shortcomings, and devise a statistically and computationally efficient estimator with near-optimal performance. These results highlight the benefits of higher-order preference data in learning correlated utilities, allowing for more fine-grained modeling of human preferences. Finally, we validate these theoretical guarantees on several real-world datasets, demonstrating improved personalization of human preferences.


'Kill Bill' slays on at Hollywood Forever Cemetery

Los Angeles Times

Misguided though it may have been to cleave "Kill Bill" in two, the decision yielded more than just a maddening exercise in delayed gratification and ruptured narrative. It separated the individual flavors, genres and tonal extremities of Quentin Tarantino's sprawling revenge epic into two distinct yin-and-yang halves, capable of being savored in isolation even when screened back-to-back. The two are connected less by narrative than by the feverish intensity of Tarantino's genre-straddling cinephilia. This Saturday's Cinespia screening at Hollywood Forever Cemetery may not be "The Whole Bloody Affair," the unofficial title of a combined single-film version (complete with extended 30-minute anime sequence) that has made the rounds in recent years. But any time there's a chance to see Uma Thurman shred every enemy in her path, well, all is right in the jungle. When: July 16, gates open at 7:15 p.m., movie starts at 9 p.m. Robot' is back, and it looks and feels like nothing else on TV Under pressure to turn his struggling studio around, will Paramount's Brad Grey survive the turmoil at Viacom?