An example of this is question answering (QA): given a question from the user, the model may provide a brief answer, but it may also follow with supporting facts and explanations, which can vary in form and detail. The user can be satisfied by a wide variety of responses, irrespective of their style or (to some extent) the choice of supporting facts included. Free-form NLG poses significant challenges to uncertainty quantification: some aspects of generation are irrelevant to the task's purpose and best excluded from uncertainty quantification, but it often appears that we are unable to characterize them precisely. If left unaddressed, however, the model's variation in the irrelevant aspects may dominate in standard uncertainty measures such as token-level entropy (Kuhn et al., 2023), making them uninformative about the model's actual performance on the task. Starting from Kuhn et al. (2023), a recent line of work (Kuhn et al., 2023; Lin et al., 2024; Zhang et al., 2023; Aichberger et al., 2024) studied this issue and proposed measuring the "semantic uncertainty" of generation; "semantics" is defined as the equivalence class of textual responses that logically entail one another. Empirical improvements in downstream tasks evidenced their contributions and highlighted the importance of task-specific uncertainty quantification, but important conceptual and practical issues remain. From a practical perspective, semantic equivalence is estimated using machine learning models, resulting in imprecise estimates that do not necessarily define an equivalence relation.

Black-box predictive models are increasingly used to generate efficient substitutes for gold-standard labels when the latter are difficult to come by. For example, predictions of protein structures are used as efficient substitutes for slow and expensive experimental measurements [3, 4, 8], and large language models are used to cheaply generate substitutes for scarce human annotations [5, 7, 14]. Prediction-powered inference (PPI) [1] is a recent framework for statistical inference that combines a large amount of machine-learning predictions with a small amount of real data to ensure simultaneously valid and statistically powerful conclusions. While PPI [1] (and its improvement PPI++ [2]) offers a principled solution to incorporating black-box predictions into the scientific workflow, its scope of application is still limited. The current analyses focus on certain convex M-estimators such as means, quantiles, and GLMs to ensure tractable implementation.

Generative Foundation Models (GFMs) have produced synthetic data with remarkable quality in modalities such as images and text. However, applying GFMs to tabular data poses significant challenges due to the inherent heterogeneity of table features. Existing cross-table learning frameworks are hindered by the absence of both a generative model backbone and a decoding mechanism for heterogeneous feature values. To overcome these limitations, we introduce the Cross-Table Synthesizer (CTSyn), a diffusion-based foundational model tailored for tabular data generation. CTSyn introduces three major components: an aggregator that consolidates heterogeneous tables into a unified latent space; a conditional latent diffusion model for sampling from this space; and type-specific decoders that reconstruct values of varied data types from sampled latent vectors. Extensive testing on real-world datasets reveals that CTSyn not only significantly outperforms existing table synthesizers in utility and diversity, but also uniquely enhances performances of downstream machine learning beyond what is achievable with real data, thus establishing a new paradigm for synthetic data generation.

Large language models (LLMs) have showcased impressive abilities across various natural language tasks. Efficiently training these massive models remains a major challenge, demanding extensive computational resources and time. To address this issue, a growing body of research has focused on developing techniques to minimize the memory footprint and accelerate LLM training without sacrificing performance. We present Spectrum, a method for selectively training the layers of an LLM based on their signal-tonoise ratio (SNR). Rooted in Random Matrix Theory, Spectrum utilizes the Marchenko-Pastur distribution to pinpoint informative layers according to their SNR. Unlike previous approaches, such as QLoRA [2], which quantizes the entire model, Spectrum strategically targets specific layers and modules for training while keeping others frozen. By concentrating computational resources on the most informative parameters, Spectrum achieves superior performance while significantly reducing training time and memory requirements compared to state-of-the-art methods. Our main contributions are as follows: We propose Spectrum, an approach for efficient LLM training that selectively trains layers based on their SNR.

Reinforcement Learning from Human Feedback (RLHF) (Christiano et al., 2017) has rapidly become a standard method to align Large Language Models (LLMs). One of the main practical issues that all the prominent existing RLHF methods (offline or online) (Ouyang et al., 2022; Rafailov et al., 2023; Azar et al., 2023; Zhao et al., 2023b; Ahmadian et al., 2024) encounter is that their optimal solution heavily depends on the training task in terms of the distribution used to generate the preference data (behavior policy) (Munos et al., 2023; Azar et al., 2023). This makes the existing RLHF methods prone to out-of-distribution (OOD) tasks (Li et al., 2024; Kirk et al., 2024) where the evaluation distribution is significantly different from that of the behavior policy. Also, whenever the base/SFT models significantly differ from the behavior policy, the dependency of the RLHF solutions on the behavior policy makes the preference dataset and reward model less useful (Gao et al., 2022) as RLHF may undo the SFT/pretraining. To address this challenge, we introduce an alternative approach for aligning LLMs from human preferences based on more principled and robust foundations. Our goal is to find a solution that is robust to the changes in the preference dataset, meaning that changes in the distribution from which the completions are sampled do not affect the final outcome of learning significantly. To achieve this goal, we exploit the concept of self-improving (Huang et al., 2022; Bai et al., 2022) language models. By self-improving LLM we refer to a model capable of enhancing its outputs recursively with each inference iteration.

Most popular benchmarks for comparing LLMs rely on a limited set of prompt templates, which may not fully capture the LLMs' abilities and can affect the reproducibility of results on leaderboards. Many recent works empirically verify prompt sensitivity and advocate for changes in LLM evaluation. In this paper, we consider the problem of estimating the performance distribution across many prompt variants instead of finding a single prompt to evaluate with. We introduce PromptEval, a method for estimating performance across a large set of prompts borrowing strength across prompts and examples to produce accurate estimates under practical evaluation budgets. The resulting distribution can be used to obtain performance quantiles to construct various robust performance metrics (e.g., top 95% quantile or median). We prove that PromptEval consistently estimates the performance distribution and demonstrate its efficacy empirically on three prominent LLM benchmarks: MMLU, BIG-bench Hard, and LMentry. For example, PromptEval can accurately estimate performance quantiles across 100 prompt templates on MMLU with a budget equivalent to two single-prompt evaluations. Our code and data can be found at https://github.com/felipemaiapolo/prompt-eval.

The natural language capabilities demonstrated by large language models (LLMs) highlight an opportunity for new writing support tools that address the varied needs of people with dyslexia. We present LaMPost, a prototype email editor that draws upon our understanding of these needs to motivate AI-powered writing features, such as outlining main ideas, generating a subject line, suggesting changes, and rewriting a selection. We evaluated LaMPost with 19 adults with dyslexia, identifying promising routes for further exploration (such as the popular "rewrite" and "subject line" features), while also finding that the current generation of LLMs may not yet meet the accuracy and quality thresholds to be useful for writers with dyslexia. In addition, knowledge of the AI did not alter participants' perception of the system nor their feelings of autonomy, expression, and self-efficacy when writing emails. Our findings provide insight into the benefits and drawbacks of LLMs as writing support for adults with dyslexia, and they offer a foundation to build upon in future research. Dyslexia refers to a cluster of symptoms that result in challenges with word recognition, reading fluency, spelling, and writing population.8

UPenn Wharton School Associate Professor Ethan Mollick weighs in on the Biden White House's new guidelines for artificial intelligence in the workplace on'Fox News Live.' Artificial intelligence systems like ChatGPT could soon run out of what keeps making them smarter -- the tens of trillions of words people have written and shared online. A new study released Thursday by research group Epoch AI projects that tech companies will exhaust the supply of publicly available training data for AI language models by roughly the turn of the decade -- sometime between 2026 and 2032. Comparing it to a "literal gold rush" that depletes finite natural resources, Tamay Besiroglu, an author of the study, said the AI field might face challenges in maintaining its current pace of progress once it drains the reserves of human-generated writing. In the short term, tech companies like ChatGPT-maker OpenAI and Google are racing to secure and sometimes pay for high-quality data sources to train their AI large language models – for instance, by signing deals to tap into the steady flow of sentences coming out of Reddit forums and news media outlets. In the longer term, there won't be enough new blogs, news articles and social media commentary to sustain the current trajectory of AI development, putting pressure on companies to tap into sensitive data now considered private -- such as emails or text messages -- or relying on less-reliable "synthetic data" spit out by the chatbots themselves.

ChatGPT developer OpenAI's approach to building artificial intelligence came under fire this week from former employees who accuse the company of taking unnecessary risks with technology that could become harmful. Today OpenAI released a new research paper apparently aimed at showing it is serious about tackling AI risk by making its models more explainable. In the paper, researchers from the company lay out a way to peer inside the AI model that powers ChatGPT. They devised a way to identify how it stores certain concepts--including those that might perhaps cause an AI system to misbehave. Although the research makes OpenAI's work on keeping AI in check more visible, it also highlights recent turmoil at the company.

Turning to a friend or coworker can make tricky problems easier to tackle. Now it looks like having AI chatbots team up with each other can make them more effective. I've been playing this week with AutoGen, an open source software framework for AI agent collaboration developed by researchers at Microsoft and academics at Pennsylvania State University, the University of Washington, and Xidian University in China. The software taps OpenAI's large language model GPT-4 to let you create multiple AI agents with different personas, roles, and objectives that can be prompted to solve specific problems. To put the idea of AI collaboration to the test, I had two AI agents work together on a plan for how to write about AI collaboration.