Government
The Distortion of Binomial Voting Defies Expectation
Gonczarowski, Yannai A., Kehne, Gregory, Procaccia, Ariel D., Schiffer, Ben, Zhang, Shirley
In computational social choice, the distortion of a voting rule quantifies the degree to which the rule overcomes limited preference information to select a socially desirable outcome. This concept has been investigated extensively, but only through a worst-case lens. Instead, we study the expected distortion of voting rules with respect to an underlying distribution over voter utilities. Our main contribution is the design and analysis of a novel and intuitive rule, binomial voting, which provides strong distribution-independent guarantees for both expected distortion and expected welfare.
Can Large Language Models Transform Computational Social Science?
Ziems, Caleb, Held, William, Shaikh, Omar, Chen, Jiaao, Zhang, Zhehao, Yang, Diyi
Large Language Models (LLMs) are capable of successfully performing many language processing tasks zero-shot (without training data). If zero-shot LLMs can also reliably classify and explain social phenomena like persuasiveness and political ideology, then LLMs could augment the Computational Social Science (CSS) pipeline in important ways. This work provides a road map for using LLMs as CSS tools. Towards this end, we contribute a set of prompting best practices and an extensive evaluation pipeline to measure the zero-shot performance of 13 language models on 25 representative English CSS benchmarks. On taxonomic labeling tasks (classification), LLMs fail to outperform the best fine-tuned models but still achieve fair levels of agreement with humans. On free-form coding tasks (generation), LLMs produce explanations that often exceed the quality of crowdworkers' gold references. We conclude that the performance of today's LLMs can augment the CSS research pipeline in two ways: (1) serving as zero-shot data annotators on human annotation teams, and (2) bootstrapping challenging creative generation tasks (e.g., explaining the underlying attributes of a text). In summary, LLMs are posed to meaningfully participate in} social science analysis in partnership with humans.
Reinforcement Learning for Combining Search Methods in the Calibration of Economic ABMs
Glielmo, Aldo, Favorito, Marco, Chanda, Debmallya, Gatti, Domenico Delli
Calibrating agent-based models (ABMs) in economics and finance typically involves a derivative-free search in a very large parameter space. In this work, we benchmark a number of search methods in the calibration of a well-known macroeconomic ABM on real data, and further assess the performance of "mixed strategies" made by combining different methods. We find that methods based on random-forest surrogates are particularly efficient, and that combining search methods generally increases performance since the biases of any single method are mitigated. Moving from these observations, we propose a reinforcement learning (RL) scheme to automatically select and combine search methods on-the-fly during a calibration run. The RL agent keeps exploiting a specific method only as long as this keeps performing well, but explores new strategies when the specific method reaches a performance plateau. The resulting RL search scheme outperforms any other method or method combination tested, and does not rely on any prior information or trial and error procedure.
Similarity of Neural Architectures using Adversarial Attack Transferability
Hwang, Jaehui, Han, Dongyoon, Heo, Byeongho, Park, Song, Chun, Sanghyuk, Lee, Jong-Seok
In recent years, many deep neural architectures have been developed for image classification. Whether they are similar or dissimilar and what factors contribute to their (dis)similarities remains curious. To address this question, we aim to design a quantitative and scalable similarity measure between neural architectures. We propose Similarity by Attack Transferability (SAT) from the observation that adversarial attack transferability contains information related to input gradients and decision boundaries widely used to understand model behaviors. We conduct a large-scale analysis on 69 state-of-the-art ImageNet classifiers using our proposed similarity function to answer the question. Moreover, we observe neural architecture-related phenomena using model similarity that model diversity can lead to better performance on model ensembles and knowledge distillation under specific conditions. Our results provide insights into why developing diverse neural architectures with distinct components is necessary.
Low-skilled Occupations Face the Highest Upskilling Pressure
Tong, Di, Wu, Lingfei, Evans, James Allen
Substantial scholarship has estimated the susceptibility of jobs to automation, but little has examined how job contents evolve in the information age as new technologies substitute for tasks, shifting required skills rather than eliminating entire jobs. Here we explore patterns and consequences of changes in occupational skill and characterize occupations and workers subject to the greatest re-skilling pressure. Recent work found that changing skill requirements are greatest for STEM occupations. Nevertheless, analyzing 167 million online job posts covering 727 occupations over the last decade, we find that re-skilling pressure is greatest for low-skilled occupations when accounting for distance between skills. We further investigate the differences in skill change across employer and market size, as well as social demographic groups, and find that these differences tend to widen the economic divide. Jobs from large employers and markets experienced less change relative to small employers and markets, and non-white workers in low-skilled jobs are most demographically vulnerable. We conclude by showcasing our model's potential to precisely chart job evolution towards machine-interface integration using skill embedding spaces.
Small Area Estimation of Case Growths for Timely COVID-19 Outbreak Detection
She, Zhaowei, Wang, Zilong, Chhatwal, Jagpreet, Ayer, Turgay
The COVID-19 pandemic has exerted a profound impact on the global economy and continues to exact a significant toll on human lives. The COVID-19 case growth rate stands as a key epidemiological parameter to estimate and monitor for effective detection and containment of the resurgence of outbreaks. A fundamental challenge in growth rate estimation and hence outbreak detection is balancing the accuracy-speed tradeoff, where accuracy typically degrades with shorter fitting windows. In this paper, we develop a machine learning (ML) algorithm, which we call Transfer Learning Generalized Random Forest (TLGRF), that balances this accuracy-speed tradeoff. Specifically, we estimate the instantaneous COVID-19 exponential growth rate for each U.S. county by using TLGRF that chooses an adaptive fitting window size based on relevant day-level and county-level features affecting the disease spread. Through transfer learning, TLGRF can accurately estimate case growth rates for counties with small sample sizes. Out-of-sample prediction analysis shows that TLGRF outperforms established growth rate estimation methods. Furthermore, we conducted a case study based on outbreak case data from the state of Colorado and showed that the timely detection of outbreaks could have been improved by up to 224% using TLGRF when compared to the decisions made by Colorado's Department of Health and Environment (CDPHE). To facilitate implementation, we have developed a publicly available outbreak detection tool for timely detection of COVID-19 outbreaks in each U.S. county, which received substantial attention from policymakers.
Towards Aligned Canonical Correlation Analysis: Preliminary Formulation and Proof-of-Concept Results
Cheng, Biqian, Papalexakis, Evangelos E., Chen, Jia
Canonical Correlation Analysis (CCA) has been widely applied to jointly embed multiple views of data in a maximally correlated latent space. However, the alignment between various data perspectives, which is required by traditional approaches, is unclear in many practical cases. In this work we propose a new framework Aligned Canonical Correlation Analysis (ACCA), to address this challenge by iteratively solving the alignment and multi-view embedding.
Kernel quadrature with randomly pivoted Cholesky
Epperly, Ethan N., Moreno, Elvira
This paper presents new quadrature rules for functions in a reproducing kernel Hilbert space using nodes drawn by a sampling algorithm known as randomly pivoted Cholesky. The resulting computational procedure compares favorably to previous kernel quadrature methods, which either achieve low accuracy or require solving a computationally challenging sampling problem. Theoretical and numerical results show that randomly pivoted Cholesky is fast and achieves comparable quadrature error rates to more computationally expensive quadrature schemes based on continuous volume sampling, thinning, and recombination. Randomly pivoted Cholesky is easily adapted to complicated geometries with arbitrary kernels, unlocking new potential for kernel quadrature.
Google says new AI model Gemini outperforms ChatGPT in most tests
Google has unveiled a new artificial intelligence model that it claims outperforms ChatGPT in most tests and displays "advanced reasoning" across multiple formats, including an ability to view and mark a student's physics homework. The model, called Gemini, is the first to be announced since last month's global AI safety summit where tech firms agreed to collaborate with governments on testing advanced systems before and after their release. Google said it was in discussions with the UK's newly formed AI Safety Institute over testing Gemini's most powerful version, which will be released next year. The model comes in three versions and is "multimodal", which means it can comprehend text, audio, images, video and computer code simultaneously. Gemini, which will be folded into Google products including its search engine, is being released initially in more than 170 countries including the US on Wednesday in the form of an upgrade to Google's chatbot Bard.
Sam Altman
It was a strange Thanksgiving for Sam Altman. Normally, the CEO of OpenAI flies home to St. Louis to visit family. But this time the holiday came after an existential struggle for control of a company that some believe holds the fate of humanity in its hands. He went to his Napa Valley ranch for a hike, then returned to San Francisco to spend a few hours with one of the board members who had just fired and reinstated him in the span of five frantic days. He put his computer away for a few hours to cook vegetarian pasta, play loud music, and drink wine with his fiancé Oliver Mulherin. "This was a 10-out-of-10 crazy thing to live through," Altman tells TIME on Nov. 30. We're speaking exactly one year after OpenAI released Chat-GPT, the most rapidly adopted tech product ever. The impact of the chatbot and its successor, GPT-4, was transformative--for the company and the world. "For many people," Altman says, 2023 was "the year that they started taking AI seriously." Born as a nonprofit research lab dedicated to building artificial intelligence for the benefit of humanity, OpenAI became an $80 billion rocket ship. Altman emerged as one of the most powerful and venerated executives in the world, the public face and leading prophet of a technological revolution. On Nov. 17, OpenAI's nonprofit board of directors fired Altman, without warning or even much in the way of explanation. The surreal maneuvering that followed made the corporate dramas of Succession seem staid. So did OpenAI's powerful investors; one even baselessly speculated that one of the directors who defenestrated Altman was a Chinese spy. The company's visionary chief scientist voted to oust his fellow co-founder, only to backtrack. Two interim CEOs came and went.