Large Language Model
Academic Case Reports Lack Diversity: Assessing the Presence and Diversity of Sociodemographic and Behavioral Factors related to Post COVID-19 Condition
Florez, Juan Andres Medina, Raza, Shaina, Lynn, Rashida, Shakeri, Zahra, Smith, Brendan T., Dolatabadi, Elham
Understanding the prevalence, disparities, and symptom variations of Post COVID-19 Condition (PCC) for vulnerable populations is crucial to improving care and addressing intersecting inequities. This study aims to develop a comprehensive framework for integrating social determinants of health (SDOH) into PCC research by leveraging NLP techniques to analyze disparities and variations in SDOH representation within PCC case reports. Following construction of a PCC Case Report Corpus, comprising over 7,000 case reports from the LitCOVID repository, a subset of 709 reports were annotated with 26 core SDOH-related entity types using pre-trained named entity recognition (NER) models, human review, and data augmentation to improve quality, diversity and representation of entity types. An NLP pipeline integrating NER, natural language inference (NLI), trigram and frequency analyses was developed to extract and analyze these entities. Both encoder-only transformer models and RNN-based models were assessed for the NER objective. Fine-tuned encoder-only BERT models outperformed traditional RNN-based models in generalizability to distinct sentence structures and greater class sparsity. Exploratory analysis revealed variability in entity richness, with prevalent entities like condition, age, and access to care, and underrepresentation of sensitive categories like race and housing status. Trigram analysis highlighted frequent co-occurrences among entities, including age, gender, and condition. The NLI objective (entailment and contradiction analysis) showed attributes like "Experienced violence or abuse" and "Has medical insurance" had high entailment rates (82.4%-80.3%), while attributes such as "Is female-identifying," "Is married," and "Has a terminal condition" exhibited high contradiction rates (70.8%-98.5%).
Each Graph is a New Language: Graph Learning with LLMs
Zhou, Huachi, Du, Jiahe, Zhou, Chuang, Yang, Chang, Xiao, Yilin, Xie, Yuxuan, Huang, Xiao
Recent efforts leverage Large Language Models (LLMs) for modeling text-attributed graph structures in node classification tasks. These approaches describe graph structures for LLMs to understand or aggregate LLM-generated textual attribute embeddings through graph structure. However, these approaches face two main limitations in modeling graph structures with LLMs. (i) Graph descriptions become verbose in describing high-order graph structure. (ii) Textual attributes alone do not contain adequate graph structure information. It is challenging to model graph structure concisely and adequately with LLMs. LLMs lack built-in mechanisms to model graph structures directly. They also struggle with complex long-range dependencies between high-order nodes and target nodes. Inspired by the observation that LLMs pre-trained on one language can achieve exceptional performance on another with minimal additional training, we propose \textbf{G}raph-\textbf{D}efined \textbf{L}anguage for \textbf{L}arge \textbf{L}anguage \textbf{M}odel (GDL4LLM). This novel framework enables LLMs to transfer their powerful language understanding capabilities to graph-structured data. GDL4LLM translates graphs into a graph language corpus instead of graph descriptions and pre-trains LLMs on this corpus to adequately understand graph structures. During fine-tuning, this corpus describes the structural information of target nodes concisely with only a few tokens. By treating graphs as a new language, GDL4LLM enables LLMs to model graph structures adequately and concisely for node classification tasks. Extensive experiments on three real-world datasets demonstrate that GDL4LLM outperforms description-based and textual attribute embeddings-based baselines by efficiently modeling different orders of graph structure with LLMs.
Med-R$^2$: Crafting Trustworthy LLM Physicians through Retrieval and Reasoning of Evidence-Based Medicine
Lu, Keer, Liang, Zheng, Pan, Da, Zhang, Shusen, Wu, Xin, Chen, Weipeng, Zhou, Zenan, Dong, Guosheng, Cui, Bin, Zhang, Wentao
In recent years, Large Language Models (LLMs) have exhibited remarkable capabilities in clinical scenarios. However, despite their potential, existing works face challenges when applying LLMs to medical settings. Strategies relying on training with medical datasets are highly cost-intensive and may suffer from outdated training data. Leveraging external knowledge bases is a suitable alternative, yet it faces obstacles such as limited retrieval precision and poor effectiveness in answer extraction. These issues collectively prevent LLMs from demonstrating the expected level of proficiency in mastering medical expertise. To address these challenges, we introduce Med-R^2, a novel LLM physician framework that adheres to the Evidence-Based Medicine (EBM) process, efficiently integrating retrieval mechanisms as well as the selection and reasoning processes of evidence, thereby enhancing the problem-solving capabilities of LLMs in healthcare scenarios and fostering a trustworthy LLM physician. Our comprehensive experiments indicate that Med-R^2 achieves a 14.87\% improvement over vanilla RAG methods and even a 3.59\% enhancement compared to fine-tuning strategies, without incurring additional training costs.
Treefix: Enabling Execution with a Tree of Prefixes
Souza, Beatriz, Pradel, Michael
The ability to execute code is a prerequisite for various dynamic program analyses. Learning-guided execution has been proposed as an approach to enable the execution of arbitrary code snippets by letting a neural model predict likely values for any missing variables. Although state-of-the-art learning-guided execution approaches, such as LExecutor, can enable the execution of a relative high amount of code, they are limited to predicting a restricted set of possible values and do not use any feedback from previous executions to execute even more code. This paper presents Treefix, a novel learning-guided execution approach that leverages LLMs to iteratively create code prefixes that enable the execution of a given code snippet. The approach addresses the problem in a multi-step fashion, where each step uses feedback about the code snippet and its execution to instruct an LLM to improve a previously generated prefix. This process iteratively creates a tree of prefixes, a subset of which is returned to the user as prefixes that maximize the number of executed lines in the code snippet. In our experiments with two datasets of Python code snippets, Treefix achieves 25% and 7% more coverage relative to the current state of the art in learning-guided execution, covering a total of 84% and 82% of all lines in the code snippets.
Multi-agent KTO: Reinforcing Strategic Interactions of Large Language Model in Language Game
Ye, Rong, Zhang, Yongxin, Zhang, Yikai, Kuang, Haoyu, Wei, Zhongyu, Sun, Peng
Achieving Artificial General Intelligence (AGI) requires AI agents that can not only make stratigic decisions but also engage in flexible and meaningful communication. Inspired by Wittgenstein's language game theory in Philosophical Investigations, we propose that language agents can learn through in-context interaction rather than traditional multi-stage frameworks that separate decision-making from language expression. Using Werewolf, a social deduction game that tests language understanding, strategic interaction, and adaptability, we develop the Multi-agent Kahneman & Tversky's Optimization (MaKTO). MaKTO engages diverse models in extensive gameplay to generate unpaired desirable and unacceptable responses, then employs KTO to refine the model's decision-making process. In 9-player Werewolf games, MaKTO achieves a 61% average win rate across various models, outperforming GPT-4o and two-stage RL agents by relative improvements of 23.0% and 10.9%, respectively. Notably, MaKTO also demonstrates human-like performance, winning 60% against expert players and showing only 49% detectability in Turing-style blind tests. These results showcase MaKTO's superior decision-making, strategic adaptation, and natural language generation in complex social deduction games.
Musk clashes with OpenAI's Altman over 500bn Stargate
Elon Musk is clashing with OpenAI CEO Sam Altman over the Stargate artificial intelligence (AI) infrastructure project touted by President Donald Trump, the latest in a feud between the two tech billionaires that started on OpenAI's board and is now testing Musk's influence with the new president. Trump on Tuesday had talked up a joint venture investing up to 500bn through a new partnership formed by OpenAI, the maker of ChatGPT, alongside Oracle and SoftBank. The new entity, Stargate, is already starting to build out data centres and the electricity generation needed for the further development of fast-evolving AI technology. Trump declared it "a resounding declaration of confidence in America's potential" under his new administration, with an initial private investment of 100bn that could reach five times that sum. But Musk, a close Trump adviser who helped bankroll his campaign and now leads a government cost-cutting initiative, questioned the value of the investment hours later.
Stargate Isn't a Victory for Trump
Late yesterday afternoon, the president of the United States transformed, very briefly, into the comms guy for a new tech company. At a press conference capping his first full day back in the White House, Donald Trump stood beside three of the most influential executives in the world--Sam Altman of OpenAI, Larry Ellison of Oracle, and Masayoshi Son of SoftBank--and announced the Stargate Project, "the largest AI infrastructure project, by far, in history." Although Trump's rhetoric may seem to suggest otherwise, Stargate is not a new federal program but rather a private venture uniting these three companies with other leaders in the AI race, such as Microsoft and Nvidia. The new company--for which Son will serve as chairman and OpenAI will be in charge of operations--will spend a planned 500 billion over the next four years to build data centers, power plants, and other such digital infrastructure in the United States, all in hopes of developing ever more advanced AI models. Trump presented Stargate as a victory for his "America First" agenda, saying that it may "lead to something that could be the biggest of all"--an apparent reference to superintelligent machines.
Reviews: Policy Continuation with Hindsight Inverse Dynamics
The paper presents a new approach for inverse dynamics learning which is extended to goal conditioned, multi-step inverse dynamics. The approach is combined with standard RL algorithms to solve multi-goal tasks such as the OpenAI Fetch environment. All reviewers liked the ideas presented in the paper and appreciated the contributions. The experiments were also well executed and the results are convincing. I am also convinced that the paper offers interesting aspects in the field of multi-goal RL and recommend this paper for a spotlight presentation.
Tech titans bicker over 500bn AI investment announced by Trump
Major tech moguls had their claws out for each other on Wednesday, hissing at their rivals over enormous pledges to invest in AI that had been announced by Donald Trump the day before. Trump announced Stargate, a 500bn project to be funded jointly by OpenAI, Oracle and Softbank, on Tuesday. During the announcement, the president was flanked by the leaders of those companies: Sam Altman, Larry Ellison and Masayoshi Son, respectively. Son is slated to be the chair of the project. Absent from the photo op was a representative from MGX, Abu Dhabi's state AI fund, another principal investor.
The future of AI is even more fossil fuels
Some of the biggest names in tech came together this week to announce "Stargate," a project they say will receive 500 billion in investment for US-based artificial intelligence infrastructure. The joint venture, spearheaded by OpenAI, Oracle, and SoftBank, aims to rapidly build out colossal new data centers crucial to future AI development. It will also prop-up new electricity plants needed to power these notoriously energy-intensive AI models. Stargate already has the blessing of newly-inaugurated president Donald Trump who this week said he has plans to "unleash" the US fossil fuel industry. Looser regulations on oil and gas extraction will make fossil fuels the obvious, cheapest choice to power Stargate's ambitious AI agenda.