Goto

Collaborating Authors

 FDA


A Weighted Prognostic Covariate Adjustment Method for Efficient and Powerful Treatment Effect Inferences in Randomized Controlled Trials

arXiv.org Machine Learning

A crucial task for a randomized controlled trial (RCT) is to specify a statistical method that can yield an efficient estimator and powerful test for the treatment effect. A novel and effective strategy to obtain efficient and powerful treatment effect inferences is to incorporate predictions from generative artificial intelligence (AI) algorithms into covariate adjustment for the regression analysis of a RCT. Training a generative AI algorithm on historical control data enables one to construct a digital twin generator (DTG) for RCT participants, which utilizes a participant's baseline covariates to generate a probability distribution for their potential control outcome. Summaries of the probability distribution from the DTG are highly predictive of the trial outcome, and adjusting for these features via regression can thus improve the quality of treatment effect inferences, while satisfying regulatory guidelines on statistical analyses, for a RCT. However, a critical assumption in this strategy is homoskedasticity, or constant variance of the outcome conditional on the covariates. In the case of heteroskedasticity, existing covariate adjustment methods yield inefficient estimators and underpowered tests. We propose to address heteroskedasticity via a weighted prognostic covariate adjustment methodology (Weighted PROCOVA) that adjusts for both the mean and variance of the regression model using information obtained from the DTG. We prove that our method yields unbiased treatment effect estimators, and demonstrate via comprehensive simulation studies and case studies from Alzheimer's disease that it can reduce the variance of the treatment effect estimator, maintain the Type I error rate, and increase the power of the test for the treatment effect from 80% to 85%~90% when the variances from the DTG can explain 5%~10% of the variation in the RCT participants' outcomes.


Challenging the Machinery of Generative AI with Fact-Checking: Ontology-Driven Biological Graphs for Verifying Human Disease-Gene Links

arXiv.org Artificial Intelligence

Background: Since the launch of various generative AI tools, scientists have been striving to evaluate their capabilities and contents, in the hope of establishing trust in their generative abilities. Regulations and guidelines are emerging to verify generated contents and identify novel uses. Objective: we aspire to demonstrate how ChatGPT claims are checked computationally using the rigor of network models. We aim to achieve fact-checking of the knowledge embedded in biological graphs that were contrived from ChatGPT contents at the aggregate level. Methods: We adopted a biological networks approach that enables the systematic interrogation of ChatGPT's linked entities. We designed an ontology-driven fact-checking algorithm that compares biological graphs constructed from approximately 200,000 PubMed abstracts with counterparts constructed from a dataset generated using the ChatGPT-3.5 Turbo model. Results: in 10-samples of 250 randomly selected records a ChatGPT dataset of 1000 "simulated" articles, the fact-checking link accuracy ranged from 70% to 86%. The computational process was followed by a manual process using IntAct Interaction database and the Gene regulatory network database (GRNdb) to confirm the validity of the links identified computationally. We also found that the proximity of the edges of ChatGPT graphs were significantly shorter (90 -- 153) while literature distances were (236 -- 765). This pattern held true in all 10-samples. Conclusion: This study demonstrated high accuracy of aggregate disease-gene links relationships found in ChatGPT-generated texts. The strikingly consistent pattern offers an illuminate new biological pathways that may open the door for new research opportunities.


Microsoft and Paige partner to create world's largest AI model for cancer detection: 'Unprecedented scale'

FOX News

Thomas Fuchs, the Dean of Artificial Intelligence and Human Health at Mount Sinai in NYC, said AI will be needed to retain the standard of care in the U.S. Microsoft is partnering with the digital pathology company Paige to build the world's largest image-based artificial intelligence (AI) model to help detect cancer, the companies announced. The AI model will be used for digital pathology and oncology, configured with billions of parameters to provide a computer vision AI that is orders of magnitude larger than any similar model existing today. Dr. Thomas Fuchs, Paige's founder and chief scientist, told FOX News Digital that the amount of data used in the model is "orders of magnitude" larger than anything made public by Google or Facebook. "It's so much larger than anything that has been published in that area ever," he said. That scale is essential for patients.


Adaptive Selection of the Optimal Strategy to Improve Precision and Power in Randomized Trials

arXiv.org Machine Learning

Benkeser et al. demonstrate how adjustment for baseline covariates in randomized trials can meaningfully improve precision for a variety of outcome types. Their findings build on a long history, starting in 1932 with R.A. Fisher and including more recent endorsements by the U.S. Food and Drug Administration and the European Medicines Agency. Here, we address an important practical consideration: *how* to select the adjustment approach -- which variables and in which form -- to maximize precision, while maintaining Type-I error control. Balzer et al. previously proposed *Adaptive Prespecification* within TMLE to flexibly and automatically select, from a prespecified set, the approach that maximizes empirical efficiency in small trials (N$<$40). To avoid overfitting with few randomized units, selection was previously limited to working generalized linear models, adjusting for a single covariate. Now, we tailor Adaptive Prespecification to trials with many randomized units. Using $V$-fold cross-validation and the estimated influence curve-squared as the loss function, we select from an expanded set of candidates, including modern machine learning methods adjusting for multiple covariates. As assessed in simulations exploring a variety of data generating processes, our approach maintains Type-I error control (under the null) and offers substantial gains in precision -- equivalent to 20-43\% reductions in sample size for the same statistical power. When applied to real data from ACTG Study 175, we also see meaningful efficiency improvements overall and within subgroups.


What Are People Asking About COVID-19? A Question Classification Dataset

arXiv.org Artificial Intelligence

We present COVID-Q, a set of 1,690 questions about COVID-19 from 13 sources, which we annotate into 15 question categories and 207 question clusters. The most common questions in our dataset asked about transmission, prevention, and societal effects of COVID, and we found that many questions that appeared in multiple sources were not answered by any FAQ websites of reputable organizations such as the CDC and FDA. We post our dataset publicly at https://github.com/JerryWeiAI/COVID-Q. For classifying questions into 15 categories, a BERT baseline scored 58.1% accuracy when trained on 20 examples per category, and for a question clustering task, a BERT + triplet loss baseline achieved 49.5% accuracy. We hope COVID-Q can help either for direct use in developing applied systems or as a domain-specific resource for model evaluation.


Synchron's BCI implants may help paralyzed patients reconnect with the world

Engadget

"We're not building a BCI to control Spotify or to watch Netflix," the CEO of medical device startup Synchron tersely told Engadget via videocall last week. "There's all this hype and excitement about BCI, about where it might go," Oxley continued. "But the reality is, what's it gonna do for patients? We describe this problem for patients, not around wanting to super-augment their brain or body, but wanting to restore the fundamental agency and autonomy that [able-bodied people] take for granted." Around 31,000 Americans currently live with Amyotrophic lateral sclerosis (ALS) with another 5,000 diagnosed every year. Nearly 300,000 Americans suffer from spinal cord paralysis, and another approximately 18,000 people join those ranks annually.


Augmenting medical image classifiers with synthetic data from latent diffusion models

arXiv.org Artificial Intelligence

While hundreds of artificial intelligence (AI) algorithms are now approved or cleared by the US Food and Drugs Administration (FDA), many studies have shown inconsistent generalization or latent bias, particularly for underrepresented populations. Some have proposed that generative AI could reduce the need for real data, but its utility in model development remains unclear. Skin disease serves as a useful case study in synthetic image generation due to the diversity of disease appearance, particularly across the protected attribute of skin tone. Here we show that latent diffusion models can scalably generate images of skin disease and that augmenting model training with these data improves performance in data-limited settings. These performance gains saturate at synthetic-to-real image ratios above 10:1 and are substantially smaller than the gains obtained from adding real images. As part of our analysis, we generate and analyze a new dataset of 458,920 synthetic images produced using several generation strategies. Our results suggest that synthetic data could serve as a force-multiplier for model development, but the collection of diverse real-world data remains the most important step to improve medical AI algorithms.


Interpreting the Mechanism of Synergism for Drug Combinations Using Attention-Based Hierarchical Graph Pooling

arXiv.org Artificial Intelligence

Synergistic drug combinations provide huge potentials to enhance therapeutic efficacy and to reduce adverse reactions. However, effective and synergistic drug combination prediction remains an open question because of the unknown causal disease signaling pathways. Though various deep learning (AI) models have been proposed to quantitatively predict the synergism of drug combinations, the major limitation of existing deep learning methods is that they are inherently not interpretable, which makes the conclusions of AI models untransparent to human experts, henceforth limiting the robustness of the model conclusion and the implementation ability of these models in real-world human--AI healthcare. In this paper, we develop an interpretable graph neural network (GNN) that reveals the underlying essential therapeutic targets and the mechanism of the synergy (MoS) by mining the sub-molecular network of great importance. The key point of the interpretable GNN prediction model is a novel graph pooling layer, a self-attention-based node and edge pool (henceforth SANEpool), that can compute the attention score (importance) of genes and connections based on the genomic features and topology. As such, the proposed GNN model provides a systematic way to predict and interpret the drug combination synergism based on the detected crucial sub-molecular network. Experiments on various well-adopted drug-synergy-prediction datasets demonstrate that (1) the SANEpool model has superior predictive ability to generate accurate synergy score prediction, and (2) the sub-molecular networks detected by the SANEpool are self-explainable and salient for identifying synergistic drug combinations.


Adderall Shortages Are Dragging On--Can Video Games Help?

WIRED

Earlier this month, facing an increasingly precarious situation, the US Food and Drug Administration (FDA) and the Drug Enforcement Administration (DEA) joined forces to address the ongoing Adderall shortage. Technically, neither organization has the power to compel pharmaceutical companies to produce mixed amphetamine salts, but in the face of skyrocketing diagnoses for attention deficit hyperactivity disorder (ADHD) in the pandemic era of telemedicine, they wanted to reassure the public that they were looking into potential alternatives to stimulant medications. In a joint statement, the agencies acknowledged that while they were actively working with the pharmaceutical industry to address the shortages, the FDA did approve a "game based digital therapeutic" to address ADHD symptoms in children back in 2020. While it's unclear whether digital therapeutics can replace stimulants entirely (they probably can't), it is clear that people want options beyond amphetamines. And this summer, digital medicine company Akili Interactive dropped the first "over-the-counter" digital therapeutic for managing ADHD symptoms in adults, using the same technology underlying their previously FDA-approved prescription video game for kids.


Explainable AI for clinical risk prediction: a survey of concepts, methods, and modalities

arXiv.org Artificial Intelligence

Recent advancements in AI applications to healthcare have shown incredible promise in surpassing human performance in diagnosis and disease prognosis. With the increasing complexity of AI models, however, concerns regarding their opacity, potential biases, and the need for interpretability. To ensure trust and reliability in AI systems, especially in clinical risk prediction models, explainability becomes crucial. Explainability is usually referred to as an AI system's ability to provide a robust interpretation of its decision-making logic or the decisions themselves to human stakeholders. In clinical risk prediction, other aspects of explainability like fairness, bias, trust, and transparency also represent important concepts beyond just interpretability. In this review, we address the relationship between these concepts as they are often used together or interchangeably. This review also discusses recent progress in developing explainable models for clinical risk prediction, highlighting the importance of quantitative and clinical evaluation and validation across multiple common modalities in clinical practice. It emphasizes the need for external validation and the combination of diverse interpretability methods to enhance trust and fairness. Adopting rigorous testing, such as using synthetic datasets with known generative factors, can further improve the reliability of explainability methods. Open access and code-sharing resources are essential for transparency and reproducibility, enabling the growth and trustworthiness of explainable research. While challenges exist, an end-to-end approach to explainability in clinical risk prediction, incorporating stakeholders from clinicians to developers, is essential for success.