Large Language Models (LLMs) have demonstrated substantial progress in biomedical and clinical applications, motivating rigorous evaluation of their ability to answer nuanced, evidence-based questions. We curate a multi-source benchmark drawing from Cochrane systematic reviews and clinical guidelines, including structured recommendations from the American Heart Association and narrative guidance used by insurers. Using GPT-4o-mini and GPT-5, we observe consistent performance patterns across sources and clinical domains: accuracy is highest on structured guideline recommendations (90%) and lower on narrative guideline and systematic review questions (60--70%). We also find a strong correlation between accuracy and the citation count of the underlying systematic reviews, where each doubling of citations is associated with roughly a 30% increase in the odds of a correct answer. Models show moderate ability to reason about evidence quality when contextual information is supplied. When we incorporate retrieval-augmented prompting, providing the gold-source abstract raises accuracy on previously incorrect items to 0.79; providing top 3 PubMed abstracts (ranked by semantic relevance) improves accuracy to 0.23, while random abstracts reduce accuracy (0.10, within temperature variation). These effects are mirrored in GPT-4o-mini, underscoring that source clarity and targeted retrieval -- not just model size -- drive performance. Overall, our results highlight both the promise and current limitations of LLMs for evidence-based clinical question answering. Retrieval-augmented prompting emerges as a useful strategy to improve factual accuracy and alignment with source evidence, while stratified evaluation by specialty and question type remains essential to understand current knowledge access and to contextualize model performance.

Whether you're a student, a journalist, or a business professional, knowing how to do high-quality research and writing using trustworthy data and sources, without giving in to the temptation of AI or ChatGPT, is a skill worth developing. As I detail in my book Writing That Gets Noticed, locating credible databases and sources and accurately vetting information can be the difference between turning a story around quickly or getting stuck with outdated information. Since I had written about getting pregnant in my forties, I knew that as long as I updated my facts and figures, and included supportive and relevant peer-reviewed research, I could pull off this story. The story ran later that day, and it led to other assignments. Here are some tips I've learned that you should consider mastering before you turn to automated tools like generative AI to handle your writing work for you.

As a patient with a family history of heart disease, Karen Moore has always been diligent about monitoring her heart health. When her primary care doctor heard something unusual during a routine examination, she sent Moore to cardiologist Mark Rabbat, MD, associate professor of medicine and radiology and director of cardiac computed tomography (CT) at Loyola Medicine. Her initial tests, including electrocardiogram (ECG), echocardiograms and magnetic resonance images (MRI) did not detect any blockages. However, Moore's symptoms never went away. "The tests didn't show anything, but I still was short of breath. I still had a difficult time exercising," she said.

The American Heart Association published the results of a trial that shows stroke survivors are twice as likely to take anti-blood clot treatments when they are using an artificial intelligence (AI) platform, compared to those receiving more traditional treatment. The AI platform, AiCure, uses software algorithms on smartphones to confirm patient identify, the medication, and if the medication was taken. Patients receive automated reminders and dosing instructions as well. Healthcare workers receive real-time data which allows for early detection of patients who are not taking their meds as scheduled. SEE: Google's DeepMind and the NHS: A glimpse of what AI means for the future of healthcare (ZDNet) This latest trial, which lasted 12 weeks and was published in the American Heart Association's journal Stroke, shows more of AI's potential.

The stories generally go like this: "a study finds drinking coffee is associated with a X% decrease in [bad health outcome]" followed shortly by "the study is observational and does not prove causation." In a new study in the American Heart Association's journal Circulation: Heart Failure, researchers found a link between drinking three or more cups of coffee a day and a decreased risk of heart failure. Led by David Kao, a cardiologist at University of Colorado School of Medicine, researchers re-examined the Framingham Heart Study (FHS), "a long-term, ongoing cardiovascular cohort study of residents of the city of Framingham, Massachusetts" that began in 1948 and has grown to include over 14,000 participants. Able to analyze massive amounts of data in a short amount of time--as well as be programmed to handle uncertainties in the data, like if a reported cup of coffee is six ounces or eight ounces--machine learning can then start to ascertain and rank which variables are most associated with incidents of heart failure, giving even observational studies more explanatory power in their findings. And indeed, when the results of the FHS machine learning analysis were compare to two other well-known studies, the Cardiovascular Heart Study (CHS) and the Atherosclerosis Risk in Communities study (ARIC), the algorithm was able "to correctly predict the relationship between coffee intake and heart failure."

Somewhere in Peter Maercklein's heartbeat was an abnormality no one could find. He survived a stroke 15 years ago, but doctors never saw anything alarming on follow-up electrocardiograms. Then, one day last fall, an artificial intelligence algorithm read his EKGs and spotted something else: a ripple in the calm that indicated an elevated risk of atrial fibrillation. Specifically, the algorithm, created by physicians at Mayo Clinic, found Maercklein had an 81.49% probability of experiencing A-fib, a quivering or irregular heartbeat that can lead to heart failure and stroke. Just days later, after Maercklein agreed to participate in a research study, a wearable Holter monitor recorded an episode of A-fib while he was walking on a treadmill.

Today, Amazon Web Services (AWS) announced that Amazon Textract, a machine learning service that quickly and easily extracts text and data from scanned documents is now eligible for healthcare workloads that require HIPAA certification. This launch builds upon the existing portfolio of AWS artificial intelligence services that are HIPAA-eligible, including Amazon Translate, Amazon Comprehend, Amazon Transcribe, Amazon Polly, Amazon SageMaker and Amazon Rekognition – that help deliver better healthcare outcomes. Healthcare providers routinely extract text and data from documents such as medical records and forms through manual data entry or simple optical character recognition (OCR) software. This is a time-consuming and often inaccurate process that produces outputs requiring extensive post-processing before it can be used by other applications. What organizations want instead is the ability to accurately identify and extract text and data from forms and tables in documents of any format and from a variety of file types and templates.

In the near future, doctors may be able to apply artificial intelligence to electrocardiogram data in order to measure overall health status, according to new research published in Circulation: Arrhythmia and Electrophysiology, a journal of the American Heart Association. An electrocardiogram, also known as an EKG or ECG, is a test used to measure the electrical activity of the heart. While it's known that a patient's sex and age could affect an EKG, researchers hypothesized that artificial intelligence could determine a patient's gender and estimate their'physiologic age' -- a measure of overall body function and health status distinct from chronological age. Using EKG data of almost 500,000 patients, a type of artificial intelligence known as a convolutional neural network was trained to find similarities among the input and output data. Once trained, the neural network was tested for accuracy on the data of an additional 275,000 patients by predicting the output when only given input data.

A machine learning (ML)-based risk calculator developed to assess an individual's long-term risk for atherosclerotic cardiovascular disease (ASCVD) identified 13 percent more high-risk patients and recommended unnecessary statin therapy 25 percent less often than standard risk assessment tools in initial tests, researchers reported in the Journal of the American Heart Association. First author Ioannis A. Kakadiaris, PhD, and colleagues with the Society for Heart Attack Prevention and Eradication (SHAPE) wrote in JAHA that the current gold standard for ASCVD risk assessment--the American College of Cardiology and American Heart Association's Pooled Cohort Equations Risk Calculator--is flawed in its accuracy. "Studies have demonstrated that the current U.S. guidelines based on the ACC/AHA risk calculator may underestimate risk of atherosclerotic CVD in certain high-risk individuals, therefore missing opportunities for intensive therapy and preventing CVD events," Kakadiaris and coauthors said. The existing approach to CVD risk assessment desperately needs an overhaul." According to a consensus report from SHAPE, comprehensive ASCVD risk assessment should include evaluation of plaque, blood and myocardial vulnerability factors if it's going to be anywhere near accurate.

Work is underway to deploy Corti, an AI system that detects heart attacks during emergency phone calls, and it could be coming to some of the biggest cities in Europe. Following plans announced earlier this year to roll Corti out in more cities, this summer the European Emergency Number Association (EENA), whose members include cities like London, Paris, Milan, and Munich, will deliver AI-powered assistance to emergency 112 operators. In initial trials, this assistance was found to identify cardiac arrest events more quickly than human operators. Emergency call centers from Seattle to Singapore also want to make Corti part of their operations, but there's no global standard for organizations working to save lives. Some are fine with the idea of deploying the AI through the cloud, while others with privacy concerns require the AI system to operate from on-premise servers.