Goto

Collaborating Authors

 Personal


Impact of Large Language Model Assistance on Patients Reading Clinical Notes: A Mixed-Methods Study

arXiv.org Artificial Intelligence

Patients derive numerous benefits from reading their clinical notes, including an increased sense of control over their health and improved understanding of their care plan. However, complex medical concepts and jargon within clinical notes hinder patient comprehension and may lead to anxiety. We developed a patient-facing tool to make clinical notes more readable, leveraging large language models (LLMs) to simplify, extract information from, and add context to notes. We prompt engineered GPT-4 to perform these augmentation tasks on real clinical notes donated by breast cancer survivors and synthetic notes generated by a clinician, a total of 12 notes with 3868 words. In June 2023, 200 female-identifying US-based participants were randomly assigned three clinical notes with varying levels of augmentations using our tool. Participants answered questions about each note, evaluating their understanding of follow-up actions and self-reported confidence. We found that augmentations were associated with a significant increase in action understanding score (0.63 $\pm$ 0.04 for select augmentations, compared to 0.54 $\pm$ 0.02 for the control) with p=0.002. In-depth interviews of self-identifying breast cancer patients (N=7) were also conducted via video conferencing. Augmentations, especially definitions, elicited positive responses among the seven participants, with some concerns about relying on LLMs. Augmentations were evaluated for errors by clinicians, and we found misleading errors occur, with errors more common in real donated notes than synthetic notes, illustrating the importance of carefully written clinical notes. Augmentations improve some but not all readability metrics. This work demonstrates the potential of LLMs to improve patients' experience with clinical notes at a lower burden to clinicians. However, having a human in the loop is important to correct potential model errors.


Challenge design roadmap

arXiv.org Artificial Intelligence

Challenges can be seen as a type of game that motivates participants to solve serious tasks. As a result, competition organizers must develop effective game rules. However, these rules have multiple objectives beyond making the game enjoyable for participants. These objectives may include solving real-world problems, advancing scientific or technical areas, making scientific discoveries, and educating the public. In many ways, creating a challenge is similar to launching a product. It requires the same level of excitement and rigorous testing, and the goal is to attract ''customers'' in the form of participants. The process begins with a solid plan, such as a competition proposal that will eventually be submitted to an international conference and subjected to peer review. Although peer review does not guarantee quality, it does force organizers to consider the impact of their challenge, identify potential oversights, and generally improve its quality. This chapter provides guidelines for creating a strong plan for a challenge. The material draws on the preparation guidelines from organizations such as Kaggle 1 , ChaLearn 2 and Tailor 3 , as well as the NeurIPS proposal template, which some of the authors contributed to.


Truth Forest: Toward Multi-Scale Truthfulness in Large Language Models through Intervention without Tuning

arXiv.org Artificial Intelligence

Despite the great success of large language models (LLMs) in various tasks, they suffer from generating hallucinations. We introduce Truth Forest, a method that enhances truthfulness in LLMs by uncovering hidden truth representations using multi-dimensional orthogonal probes. Specifically, it creates multiple orthogonal bases for modeling truth by incorporating orthogonal constraints into the probes. Moreover, we introduce Random Peek, a systematic technique considering an extended range of positions within the sequence, reducing the gap between discerning and generating truth features in LLMs. By employing this approach, we improved the truthfulness of Llama-2-7B from 40.8\% to 74.5\% on TruthfulQA. Likewise, significant improvements are observed in fine-tuned models. We conducted a thorough analysis of truth features using probes. Our visualization results show that orthogonal probes capture complementary truth-related features, forming well-defined clusters that reveal the inherent structure of the dataset.


ChatGPT's FarmVille Moment

The Atlantic - Technology

ChatGPT has certainly captured the world's imagination since its release at the end of 2022. But in day-to-day life, it is still a relatively niche product--a curiosity that leads people to ask questions that begin "Have you tried โ€ฆ?" or "What do you think about โ€ฆ?" Its maker, OpenAI, has a much more expansive vision. Its aim is seemingly to completely remake how people use the internet. For that to happen, the bot needs to be more than a conversation starter: It has to be a functioning business.


What should I say? -- Interacting with AI and Natural Language Interfaces

arXiv.org Artificial Intelligence

As Artificial Intelligence (AI) technology becomes more and more prevalent, it becomes increasingly important to explore how we as humans interact with AI. The Human-AI Interaction (HAI) sub-field has emerged from the Human-Computer Interaction (HCI) field and aims to examine this very notion. Many interaction patterns have been implemented without fully understanding the changes in required cognition as well as the cognitive science implications of using these alternative interfaces that aim to be more human-like in nature. Prior research suggests that theory of mind representations are crucial to successful and effortless communication, however very little is understood when it comes to how theory of mind representations are established when interacting with AI.


Topology-Driven Parallel Trajectory Optimization in Dynamic Environments

arXiv.org Artificial Intelligence

Ground robots navigating in complex, dynamic environments must compute collision-free trajectories to avoid obstacles safely and efficiently. Nonconvex optimization is a popular method to compute a trajectory in real-time. However, these methods often converge to locally optimal solutions and frequently switch between different local minima, leading to inefficient and unsafe robot motion. In this work, We propose a novel topology-driven trajectory optimization strategy for dynamic environments that plans multiple distinct evasive trajectories to enhance the robot's behavior and efficiency. A global planner iteratively generates trajectories in distinct homotopy classes. These trajectories are then optimized by local planners working in parallel. While each planner shares the same navigation objectives, they are locally constrained to a specific homotopy class, meaning each local planner attempts a different evasive maneuver. The robot then executes the feasible trajectory with the lowest cost in a receding horizon manner. We demonstrate, on a mobile robot navigating among pedestrians, that our approach leads to faster and safer trajectories than existing planners.


Safe reinforcement learning in uncertain contexts

arXiv.org Artificial Intelligence

When deploying machine learning algorithms in the real world, guaranteeing safety is an essential asset. Existing safe learning approaches typically consider continuous variables, i.e., regression tasks. However, in practice, robotic systems are also subject to discrete, external environmental changes, e.g., having to carry objects of certain weights or operating on frozen, wet, or dry surfaces. Such influences can be modeled as discrete context variables. In the existing literature, such contexts are, if considered, mostly assumed to be known. In this work, we drop this assumption and show how we can perform safe learning when we cannot directly measure the context variables. To achieve this, we derive frequentist guarantees for multi-class classification, allowing us to estimate the current context from measurements. Further, we propose an approach for identifying contexts through experiments. We discuss under which conditions we can retain theoretical guarantees and demonstrate the applicability of our algorithm on a Furuta pendulum with camera measurements of different weights that serve as contexts.


I Asked Smile Experts to Analyze Ron DeSantis' Smile. I Do Not Have Good News.

Slate

Over the past few months, many have attempted to translate the uncanniness of Gov. Ron DeSantis' smile into words. After the Republican debates, it's been called "painfully weird" and said to look "like it's on his face upside down." It resembles "a Disney World animatronic" or "an A.I. trying to learn human emotions." It even inspired The Daily Show to put out a public service announcement about "Frownington's Disease," a made-up condition that causes a person's smile to resemble a wince one would make upon "sitting on his own testicles." As nice as it is that one expression has inspired such rich verbiage and creativity--Ron DeSantis, unlikely muse!--you might find yourself longing for a more technical explanation.


I Can Get Any Woman I Want Online. Somehow That Doesn't Work In Person.

Slate

How to Do It is Slate's sex advice column. Send it to Stoya and Rich here. As a sexually dominant-leaning female, I get a lot of instant gratification out of gorgeous women online telling me my assertiveness is impressive and sexy. When I have sex with women in my dreams, it's perfect. While my "traditional" long-term relationships have been with male-presenting people, I slept with several women in my early 20s--though I struggled to find satisfying connections.


If There Are No Stupid Questions, Then How Do You Explain Quora?

The Atlantic - Technology

This article was featured in the One Story to Read Today newsletter. Every day or two for the past seven months, I've received a "personalized" email containing a bunch of recent, user-generated questions from the website Quora. "I caught my son playing his Xbox at 12:00 in the morning on a school night. As a result, I broke his console and now he won't talk to me. How can I tell him that it is his fault?"