South America
Assessing Racial Disparities in Healthcare Expenditures Using Causal Path-Specific Effects
Ou, Xiaxian, He, Xinwei, Benkeser, David, Nabi, Razieh
Racial disparities in healthcare expenditures are well-documented, yet the underlying drivers remain complex and require further investigation. This study employs causal and counterfactual path-specific effects to quantify how various factors, including socioeconomic status, insurance access, health behaviors, and health status, mediate these disparities. Using data from the Medical Expenditures Panel Survey, we estimate how expenditures would differ under counterfactual scenarios in which the values of specific mediators were aligned across racial groups along selected causal pathways. A key challenge in this analysis is ensuring robustness against model misspecification while addressing the zero-inflation and right-skewness of healthcare expenditures. For reliable inference, we derive asymptotically linear estimators by integrating influence function-based techniques with flexible machine learning methods, including super learners and a two-part model tailored to the zero-inflated, right-skewed nature of healthcare expenditures.
Automatic Legal Writing Evaluation of LLMs
Pires, Ramon, Junior, Roseval Malaquias, Nogueira, Rodrigo
Despite the recent advances in Large Language Models, benchmarks for evaluating legal writing remain scarce due to the inherent complexity of assessing open-ended responses in this domain. One of the key challenges in evaluating language models on domain-specific tasks is finding test datasets that are public, frequently updated, and contain comprehensive evaluation guidelines. The Brazilian Bar Examination meets these requirements. We introduce oab-bench, a benchmark comprising 105 questions across seven areas of law from recent editions of the exam. The benchmark includes comprehensive evaluation guidelines and reference materials used by human examiners to ensure consistent grading. We evaluate the performance of four LLMs on oab-bench, finding that Claude-3.5 Sonnet achieves the best results with an average score of 7.93 out of 10, passing all 21 exams. We also investigated whether LLMs can serve as reliable automated judges for evaluating legal writing. Our experiments show that frontier models like OpenAI's o1 achieve a strong correlation with human scores when evaluating approved exams, suggesting their potential as reliable automated evaluators despite the inherently subjective nature of legal writing assessment. The source code and the benchmark -- containing questions, evaluation guidelines, model-generated responses, and their respective automated evaluations -- are publicly available.
DOGE Put a College Student in Charge of Using AI to Rewrite Regulations
A young man with no government experience who has yet to even complete his undergraduate degree is working for Elon Musk's so-called Department of Government Efficiency (DOGE) at the Department of Housing and Urban Development (HUD) and has been tasked with using artificial intelligence to rewrite the agency's rules and regulations. Christopher Sweet was introduced to HUD employees as being originally from San Francisco and most recently a third-year at the University of Chicago, where he was studying economics and data science, in an email sent to staffers earlier this month. "I'd like to share with you that Chris Sweet has joined the HUD DOGE team with the title of special assistant, although a better title might be'Al computer programming quant analyst,'" Scott Langmack, a DOGE staffer and chief operating officer of an AI real estate company, wrote in an email widely shared within the agency and reviewed by WIRED. "With family roots from Brazil, Chris speaks Portuguese fluently. Please join me in welcoming Chris to HUD!" Sweet's primary role appears to be leading an effort to leverage artificial intelligence to review HUD's regulations, compare them to the laws on which they are based, and identify areas where rules can be relaxed or removed altogether.
'Bella the robot waitress won't replace our staff'
'Bella the robot waitress won't replace our staff' 4 days agoShareSaveSophie CridlandReporting fromPortlandShareSaveBBCMike Deadman, from The View Cafe and Bar, said Bella was not being used to replace staff Bella carries multiple trays packed with food and drinks, deftly swerving any obstacles and delivering orders day in and day out to her customers. This is the latest recruit at The View Cafe and Bar at Portland's Heights hotel in Dorset. But Bella is no normal member of the waiting staff - she is a state-of-the art robot programmed to serve and even interact with the eatery's patrons. And costing a little under 9,000, it is hoped it can be an economical idea, as well as a novel one. But assistant manager Mike Deadman insists Bella - built by Chinese technology company Pudu - will not result in any job losses.
Kaleidoscope: In-language Exams for Massively Multilingual Vision Evaluation
Salazar, Israfel, Burda, Manuel Fernández, Islam, Shayekh Bin, Moakhar, Arshia Soltani, Singh, Shivalika, Farestam, Fabian, Romanou, Angelika, Boiko, Danylo, Khullar, Dipika, Zhang, Mike, Krzemiński, Dominik, Novikova, Jekaterina, Shimabucoro, Luísa, Imperial, Joseph Marvin, Maheshwary, Rishabh, Duwal, Sharad, Amayuelas, Alfonso, Rajwal, Swati, Purbey, Jebish, Ruby, Ahmed, Popovič, Nicholas, Suppa, Marek, Wasi, Azmine Toushik, Kadiyala, Ram Mohan Rao, Tsymboi, Olga, Kostritsya, Maksim, Moakhar, Bardia Soltani, Merlin, Gabriel da Costa, Coletti, Otávio Ferracioli, Shiviari, Maral Jabbari, fard, MohammadAmin farahani, Fernandez, Silvia, Grandury, María, Abulkhanov, Dmitry, Sharma, Drishti, De Mitri, Andre Guarnier, Marchezi, Leticia Bossatto, Heydari, Setayesh, Obando-Ceron, Johan, Kohut, Nazar, Ermis, Beyza, Elliott, Desmond, Ferrante, Enzo, Hooker, Sara, Fadaee, Marzieh
The evaluation of vision-language models (VLMs) has mainly relied on English-language benchmarks, leaving significant gaps in both multilingual and multicultural coverage. While multilingual benchmarks have expanded, both in size and languages, many rely on translations of English datasets, failing to capture cultural nuances. In this work, we propose Kaleidoscope, as the most comprehensive exam benchmark to date for the multilingual evaluation of vision-language models. Kaleidoscope is a large-scale, in-language multimodal benchmark designed to evaluate VLMs across diverse languages and visual inputs. Kaleidoscope covers 18 languages and 14 different subjects, amounting to a total of 20,911 multiple-choice questions. Built through an open science collaboration with a diverse group of researchers worldwide, Kaleidoscope ensures linguistic and cultural authenticity. We evaluate top-performing multilingual vision-language models and find that they perform poorly on low-resource languages and in complex multimodal scenarios. Our results highlight the need for progress on culturally inclusive multimodal evaluation frameworks.
Multi-view autoencoders for Fake News Detection
Pereira, Ingryd V. S. T., Cavalcanti, George D. C., Cruz, Rafael M. O.
Given the volume and speed at which fake news spreads across social media, automatic fake news detection has become a highly important task. However, this task presents several challenges, including extracting textual features that contain relevant information about fake news. Research about fake news detection shows that no single feature extraction technique consistently outperforms the others across all scenarios. Nevertheless, different feature extraction techniques can provide complementary information about the textual data and enable a more comprehensive representation of the content. This paper proposes using multi-view autoencoders to generate a joint feature representation for fake news detection by integrating several feature extraction techniques commonly used in the literature. Experiments on fake news datasets show a significant improvement in classification performance compared to individual views (feature representations). We also observed that selecting a subset of the views instead of composing a latent space with all the views can be advantageous in terms of accuracy and computational effort. For further details, including source codes, figures, and datasets, please refer to the project's repository: https://github.com/ingrydpereira/multiview-fake-news.
The use of Multi-domain Electroencephalogram Representations in the building of Models based on Convolutional and Recurrent Neural Networks for Epilepsy Detection
Anghinoni, Luiz Antonio Nicolau, Denardin, Gustavo Weber, Gertrudes, Jadson Castro, Casanova, Dalcimar, Oliva, Jefferson Tales
This important role has led researchers to develop various methods for gathering information about brain activity, resulting in significant advancements in medical signal and image acquisition systems [2]. Among these advancements are functional neuroimaging techniques, such as functional magnetic resonance imaging, magnetoencephalography (MEG), positron emission tomography (PET), and electroencephalography [2]. Among these techniques, electroencephalography stands out due to three key advantages: it is a non-invasive method that allows data generation from any individual, has excellent temporal resolution--effectively capturing events occurring within milliseconds--and is relatively cost-effective compared to other examinations [3]. Electroencephalography monitors the brain's electrical activity through electrodes placed on the scalp, and the resulting data, known as the electroencephalogram (EEG), consists of a time series of electrical potentials that reflect neurological activity [4]. The EEG signal is widely used in the field of neuroscience and has the potential to advance brain-computer interfaces [5], facilitate emotion detection [6], enable classification of sleep stages [7] and help clinicians and researchers in identifying brain diseases, including but not limited to Alzheimer's disease [8], dyslexia [9], schizophrenia [10], Creutzfeldt-Jakob disease [11] and cognitive impairment [12]. Epilepsy, for example, is a neurological disorder characterized by abnormal brain activity that can lead to seizures, unusual behaviors, or even loss of consciousness.
AI Alignment in Medical Imaging: Unveiling Hidden Biases Through Counterfactual Analysis
Ma, Haroui, Quinzan, Francesco, Willem, Theresa, Bauer, Stefan
Machine learning (ML) systems for medical imaging have demonstrated remarkable diagnostic capabilities, but their susceptibility to biases poses significant risks, since biases may negatively impact generalization performance. In this paper, we introduce a novel statistical framework to evaluate the dependency of medical imaging ML models on sensitive attributes, such as demographics. Our method leverages the concept of counterfactual invariance, measuring the extent to which a model's predictions remain unchanged under hypothetical changes to sensitive attributes. We present a practical algorithm that combines conditional latent diffusion models with statistical hypothesis testing to identify and quantify such biases without requiring direct access to counterfactual data. Through experiments on synthetic datasets and large-scale real-world medical imaging datasets, including \textsc{cheXpert} and MIMIC-CXR, we demonstrate that our approach aligns closely with counterfactual fairness principles and outperforms standard baselines. This work provides a robust tool to ensure that ML diagnostic systems generalize well, e.g., across demographic groups, offering a critical step towards AI safety in healthcare. Code: https://github.com/Neferpitou3871/AI-Alignment-Medical-Imaging.
On Stopping Times of Power-one Sequential Tests: Tight Lower and Upper Bounds
Agrawal, Shubhada, Ramdas, Aaditya
We prove two lower bounds for stopping times of sequential tests between general composite nulls and alternatives. The first lower bound is for the setting where the type-1 error level $\alpha$ approaches zero, and equals $\log(1/\alpha)$ divided by a certain infimum KL divergence, termed $\operatorname{KL_{inf}}$. The second lower bound applies to the setting where $\alpha$ is fixed and $\operatorname{KL_{inf}}$ approaches 0 (meaning that the null and alternative sets are not separated) and equals $c \operatorname{KL_{inf}}^{-1} \log \log \operatorname{KL_{inf}}^{-1}$ for a universal constant $c > 0$. We also provide a sufficient condition for matching the upper bounds and show that this condition is met in several special cases. Given past work, these upper and lower bounds are unsurprising in their form; our main contribution is the generality in which they hold, for example, not requiring reference measures or compactness of the classes.
On learning functions over biological sequence space: relating Gaussian process priors, regularization, and gauge fixing
Petti, Samantha, Martí-Gómez, Carlos, Kinney, Justin B., Zhou, Juannan, McCandlish, David M.
Mappings from biological sequences (DNA, RNA, protein) to quantitative measures of sequence functionality play an important role in contemporary biology. We are interested in the related tasks of (i) inferring predictive sequence-to-function maps and (ii) decomposing sequence-function maps to elucidate the contributions of individual subsequences. Because each sequence-function map can be written as a weighted sum over subsequences in multiple ways, meaningfully interpreting these weights requires "gauge-fixing," i.e., defining a unique representation for each map. Recent work has established that most existing gauge-fixed representations arise as the unique solutions to $L_2$-regularized regression in an overparameterized "weight space" where the choice of regularizer defines the gauge. Here, we establish the relationship between regularized regression in overparameterized weight space and Gaussian process approaches that operate in "function space," i.e. the space of all real-valued functions on a finite set of sequences. We disentangle how weight space regularizers both impose an implicit prior on the learned function and restrict the optimal weights to a particular gauge. We also show how to construct regularizers that correspond to arbitrary explicit Gaussian process priors combined with a wide variety of gauges. Next, we derive the distribution of gauge-fixed weights implied by the Gaussian process posterior and demonstrate that even for long sequences this distribution can be efficiently computed for product-kernel priors using a kernel trick. Finally, we characterize the implicit function space priors associated with the most common weight space regularizers. Overall, our framework unifies and extends our ability to infer and interpret sequence-function relationships.