clinical review
Automated Paper Screening for Clinical Reviews Using Large Language Models
Guo, Eddie, Gupta, Mehul, Deng, Jiawen, Park, Ye-Jean, Paget, Mike, Naugler, Christopher
Objective: To assess the performance of the OpenAI GPT API in accurately and efficiently identifying relevant titles and abstracts from real-world clinical review datasets and compare its performance against ground truth labelling by two independent human reviewers. Methods: We introduce a novel workflow using the OpenAI GPT API for screening titles and abstracts in clinical reviews. A Python script was created to make calls to the GPT API with the screening criteria in natural language and a corpus of title and abstract datasets that have been filtered by a minimum of two human reviewers. We compared the performance of our model against human-reviewed papers across six review papers, screening over 24,000 titles and abstracts. Results: Our results show an accuracy of 0.91, a sensitivity of excluded papers of 0.91, and a sensitivity of included papers of 0.76. On a randomly selected subset of papers, the GPT API demonstrated the ability to provide reasoning for its decisions and corrected its initial decision upon being asked to explain its reasoning for a subset of incorrect classifications. Conclusion: The GPT API has the potential to streamline the clinical review process, save valuable time and effort for researchers, and contribute to the overall quality of clinical reviews. By prioritizing the workflow and acting as an aid rather than a replacement for researchers and reviewers, the GPT API can enhance efficiency and lead to more accurate and reliable conclusions in medical research.
How artificial intelligence can save health insurers $7 billion, Accenture says
Insurers can save up to $7 billion over 18 months using technologies driven by artificial intelligence, according to an Accenture report. About three-quarters, or 72 percent of payer executives, said within the year, AI will be one of their top three strategic priorities for their organization. Accenture identified six areas where AI can make a difference in an insurer's operating model, and said the top three are in anticipating and resolving customer questions, improving the benefits loading and design process and accelerating prior authorization and clinical review of claims. Money saved by using AI in the six areas include $2.1 billion in managing customer interactions; $1.4 billion in managing membership and billing; $1.1 billion in managing and support reimbursement by automating claims processing and reviews; $1 billion in managing network and providers; $.9 billion in performing health management to engage members in improving outcomes with intelligent solutions; and $.5 billion in managing quality improvement and compliance via automated reporting and regulatory updates. In workforce management, the result of automating core administrative functions using AI equates to unlocking $15 million in operating income for every 100 full-time employees, the report said.