Discharge Summaries are documents written by medical professionals that detail a patient's visit to a care facility. They contain a wealth of information crucial for patient care, and automating their generation could significantly reduce the effort required from healthcare professionals, minimize errors, and ensure that critical patient information is easily accessible and actionable. In this work, we explore the use of five Large Language Models on this task, from open-source models (Mistral, Llama 2) to proprietary systems (GPT-3, GPT-4, Gemini 1.5 Pro), leveraging MIMIC-III summaries and notes. We evaluate them using exact-match, soft-overlap, and reference-free metrics. Our results show that proprietary models, particularly Gemini with one-shot prompting, outperformed others, producing summaries with the highest similarity to the gold-standard ones. Open-source models, while promising, especially Mistral after fine-tuning, lagged in performance, often struggling with hallucinations and repeated information. Human evaluation by a clinical expert confirmed the practical utility of the summaries generated by proprietary models. Despite the challenges, such as hallucinations and missing information, the findings suggest that LLMs, especially proprietary models, are promising candidates for automatic discharge summary generation as long as data privacy is ensured.

As AI systems become more capable of complex agentic tasks, they also become more capable of pursuing undesirable objectives and causing harm. Previous work has attempted to catch these unsafe instances by interrogating models directly about their objectives and behaviors. However, the main weakness of trusting interrogations is that models can lie. We propose self-report fine-tuning (SRFT), a simple supervised fine-tuning technique that trains models to occasionally make factual mistakes, then admit them when asked. We show that the admission of factual errors in simple question-answering settings generalizes out-of-distribution (OOD) to the admission of hidden misaligned objectives in adversarial agentic settings. We evaluate SRFT in OOD stealth tasks, where models are instructed to complete a hidden misaligned objective alongside a user-specified objective without being caught by monitoring. After SRFT, models are more likely to confess the details of their hidden objectives when interrogated, even under strong pressure not to disclose them. Interrogation on SRFT models can detect hidden objectives with near-ceiling performance (F1 score = 0.98), while the baseline model lies when interrogated under the same conditions (F1 score = 0). Interrogation on SRFT models can further elicit the content of the hidden objective, recovering 28-100% details, compared to 0% details recovered in the baseline model and by prefilled assistant turn attacks. This provides a promising technique for promoting honesty propensity and incriminating misaligned AIs.

National Guard soldiers shot in Washington DC'terror attack' are named... as Afghan gunman's chilling 40-hour drive across the country emerges Patrick Mahomes' bizarre addition to his Thanksgiving meal leaves fans disgusted My book on the Kennedys was used as a'mistress manual' by Olivia Nuzzi... then this wannabe Carolyn Bessette had the nerve to hound me with these outrageous texts: MAUREEN CALLAHAN Bryan Kohberger becomes nightmare prison diva... as he throws huge tantrum over BANANAS behind bars Campbell's under fire again just a day after top exec was canned for soup ingredient bombshell Watch distressing moment'drugged' Tara Reid COLLAPSES... as friends reveal urgent developments about'surveillance footage': 'They should be prosecuted' Deaths from highly infectious virus are growing... as states brace for widespread outbreaks Karoline Leavitt's family member was swarmed by ICE agents while picking up son from school as child's father tell her to'self deport' Cuckolded ex of writer who'had affair' with RFK Jr rebounds with lookalike feminist... who hates men but fetches his coffee How my bifold doors nearly killed me. It's a middle-class status symbol, but a horrific danger landed me in surgery. Her people tell me extraordinary accusation is'highly defamatory'. Princess of Wales visits the Anna Freud Centre to launch new project supporting children's mental health Campbell's under fire again just a day after top exec was canned for soup ingredient bombshell READ MORE: Campbell's FIRES executive secretly recorded saying its soups are full of'bioengineered meat' and made for'poor people' Campbell's admission to illegally discharging wastewater into a major US river has resurfaced amid a scandal involving its iconic soup ingredients. The New Jersey-based company admitted in September that its Napoleon, Ohio, canning plant illegally dumped wastewater more than 5,400 times from April 2018 to December 2024, breaking federal water pollution laws.

Accordingly, the word embeddings are learned to inherit those relationships.

EHRNoteQA in two formats: open-ended and multi-choice question answering, and propose a reliable evaluation method for each. We evaluate 27 LLMs using EHRNoteQA and examine various factors affecting the model performance ( e.g., the length and number of discharge summaries).

A.1 Motivation For what purpose was the dataset created?

This dataset covers various topics and is suitable for instruction-tuning general-purpose LLMs for diverse clinical use cases.

Large Language Models have advanced clinical text classification, but their opaque predictions remain a critical barrier to practical adoption in research and clinical settings where investigators and physicians need to understand which parts of a patient's record drive risk signals. To address this challenge, we introduce \textbf{CALM}, short for \textbf{Classification with Additive Large Language Models}, an interpretable framework for semi-structured text where inputs are composed of semantically meaningful components, such as sections of an admission note or question-answer fields from an intake form. CALM predicts outcomes as the additive sum of each component's contribution, making these contributions part of the forward computation itself and enabling faithful explanations at both the patient and population level. The additive structure also enables clear visualizations, such as component-level risk curves similar to those used in generalized additive models, making the learned relationships easier to inspect and communicate. Although CALM expects semi-structured inputs, many clinical documents already have this form, and similar structure can often be automatically extracted from free-text notes. CALM achieves performance comparable to conventional LLM classifiers while improving trust, supporting quality-assurance checks, and revealing clinically meaningful patterns during model development and auditing.