Large language models (LLMs) have shown considerable promise in clinical natural language processing, yet few domain-specific datasets exist to rigorously evaluate their performance on radiology tasks. In this work, we introduce an annotated corpus of 6,393 radiology reports from 586 patients, each labeled for follow-up imaging status, to support the development and benchmarking of follow-up adherence detection systems. Using this corpus, we systematically compared traditional machine-learning classifiers, including logistic regression (LR), support vector machines (SVM), Longformer, and a fully fine-tuned Llama3-8B-Instruct, with recent generative LLMs. To evaluate generative LLMs, we tested GPT-4o and the open-source GPT-OSS-20B under two configurations: a baseline (Base) and a task-optimized (Advanced) setting that focused inputs on metadata, recommendation sentences, and their surrounding context. A refined prompt for GPT-OSS-20B further improved reasoning accuracy. Performance was assessed using precision, recall, and F1 scores with 95% confidence intervals estimated via non-parametric bootstrapping. Inter-annotator agreement was high (F1 = 0.846). GPT-4o (Advanced) achieved the best performance (F1 = 0.832), followed closely by GPT-OSS-20B (Advanced; F1 = 0.828). LR and SVM also performed strongly (F1 = 0.776 and 0.775), underscoring that while LLMs approach human-level agreement through prompt optimization, interpretable and resource-efficient models remain valuable baselines.

The annual report keeps track, collects e displays AI-related data, to support meaningful decisions, and advance AI responsibly and ethically. The AI Index Report supports many different organizations to track progress in artificial intelligence. These organizations include: the Center for Security and Emerging Technology at Georgetown University, LinkedIn, NetBase Quid, Lightcast, and McKinsey. The AI Index Report also expanded its tracking of global AI legislation from 25 countries in 2022 to 127 in 2023. The demand for AI-related job skills is increasing in virtually all industries (in the US).

The 2022 AI Index Report has been published. Compiled by the Stanford Institute for Human-Centered Artificial Intelligence (HAI), it tracks, summarises and visualises data relating to artificial intelligence. The aim of the report is to provide unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, executives, journalists, and the general public to develop intuitions about the complex field of AI. Find out more about the report here. You can access the full pdf version here.

Benefitfocus, Inc. (BNFT), a leading cloud-based benefits management platform and services provider, today announced it will present findings and recommendations from its recent Consumer Benefits Coverage Index report at the SHRM Annual Conference & Exposition. The conference is the largest gathering of human resources (HR) professionals, attracting over 18,000 attendees in Las Vegas from June 23-26, 2019. Based on a sample of one million consumers on the Benefitfocus platform which serves over 25 million individuals, the Consumer Benefits Coverage Index is a measure of how well the typical American's employee benefits portfolio aligns with their expected coverage needs. Co-presenting from Benefitfocus will be Cindi Van Meir, Director of Brand Marketing and Misty Guinn, Director of Benefits & Wellness. The session takes place on the opening day of the conference, Sunday, June 23, 2019, from 4:20-4:50 PM in the HR Technology Solutions Theater in the Las Vegas Convention Center.