Human label variation (HLV) is a valuable source of information that arises when multiple human annotators provide different labels for valid reasons. In Natural Language Inference (NLI) earlier approaches to capturing HLV involve either collecting annotations from many crowd workers to represent human judgment distribution (HJD) or use expert linguists to provide detailed explanations for their chosen labels. While the former method provides denser HJD information, obtaining it is resource-intensive. In contrast, the latter offers richer textual information but it is challenging to scale up to many human judges. Besides, large language models (LLMs) are increasingly used as evaluators (``LLM judges'') but with mixed results, and few works aim to study HJDs. This study proposes to exploit LLMs to approximate HJDs using a small number of expert labels and explanations. Our experiments show that a few explanations significantly improve LLMs' ability to approximate HJDs with and without explicit labels, thereby providing a solution to scale up annotations for HJD. However, fine-tuning smaller soft-label aware models with the LLM-generated model judgment distributions (MJDs) presents partially inconsistent results: while similar in distance, their resulting fine-tuned models and visualized distributions differ substantially. We show the importance of complementing instance-level distance measures with a global-level shape metric and visualization to more effectively evaluate MJDs against human judgment distributions.

Sen. Elizabeth WarrenElizabeth Ann WarrenTrump says his Doral resort will no longer host G-7 after backlash Ocasio-Cortez: Sanders' heart attack was a'gut check' moment Ocasio-Cortez tweets endorsement of Sanders MORE (D-Mass) and entrepreneur Andrew YangAndrew YangYang cautions Democrats: Impeachment might not be'successful' Yang defends Gabbard: She'deserves much more respect' Super PAC seeks to spend more than million supporting Yang MORE's fight over jobs and automation at the last Democratic debate highlighted the divide over the contentious issue, including among experts. Warren and Yang sparred at Tuesday's presidential debate over whether automation or trade were primarily responsible for eliminating jobs in key parts of the country. Yang's campaign is centered around a universal basic income, which would pay every adult citizen $1,000 a month to combat the job loss brought on by automation. Americans, he said, are already seeing the effects of speedy technological advancement. "Their Main Street stores are closing. They see a self-serve kiosk in every McDonalds, every grocery store, every CVS," he said at the debate in Ohio.

NEW DELHI: In a first for Indian carriers, Air India is rationing the number of alcoholic drinks being served to its premium international passengers at airport lounges across India. The AI move follows a recent plea by several Indian airlines to aviation authorities to restrict passengers' access to liquor at departure terminals to check increasing incidents of unruly behaviour by tipsy flyers. Tagged In Artificial Intelligence India Airport Beer Kolkata Chhatrapati Shivaji International Airport Air India Indira Gandhi International Airport Wine Gin Rum Chennai International Airport Indian (airline) Go Air Indi Go Airport Lounge Rajiv Gandhi International Airport Recently actually had the time to visit a very controversial exhibition in Palazzo Strozzi, by Chinese artist Ai WeiWei. Apple has given its clearest indication yet that it's working on a self-driving car – or at least working with car manufacturers to make the plans a reality. Tagged In The Wall Street Journal Wired (magazine) Apple Inc Washington, D C Minivan Mc Laren Machine Learning National Highway Traffic Safety Administration Victoria (australia) Financial Times Mc Laren Technology Group Paul Krugman would stand in line to meet Charlie Stross.