adverse event report
Critical appraisal of artificial intelligence for rare-event recognition: principles and pharmacovigilance case studies
Noren, G. Niklas, Meldau, Eva-Lisa, Ellenius, Johan
Many high-stakes AI applications target low-prevalence events, where apparent accuracy can conceal limited real-world value. Relevant AI models range from expert-defined rules and traditional machine learning to generative LLMs constrained for classification. We outline key considerations for critical appraisal of AI in rare-event recognition, including problem framing and test set design, prevalence-aware statistical evaluation, robustness assessment, and integration into human workflows. In addition, we propose an approach to structured case-level examination (SCLE), to complement statistical performance evaluation, and a comprehensive checklist to guide procurement or development of AI models for rare-event recognition. We instantiate the framework in pharmacovigilance, drawing on three studies: rule-based retrieval of pregnancy-related reports; duplicate detection combining machine learning with probabilistic record linkage; and automated redaction of person names using an LLM. We highlight pitfalls specific to the rare-event setting including optimism from unrealistic class balance and lack of difficult positive controls in test sets - and show how cost-sensitive targets align model performance with operational value. While grounded in pharmacovigilance practice, the principles generalize to domains where positives are scarce and error costs may be asymmetric.
The Growing Symbiosis Between RPA and AI
Given their inherent similarities, it's entirely logical that robotic process automation (RPA) and artificial intelligence (AI) would have crossover in a variety of different contexts. Both of these technologies are contingent upon bringing order to processes – usually workflows, in either the physical or digital realm – that might otherwise run the risk of falling into disorder for any number of reasons, most notably human error. RPA and AI are perfectly capable of operating on their own, and in fact often do. Organisations considering implementing one or the other to manage any of their operations would serve themselves well by examining how they're being used in tandem. It's also worth investigating how a versatile low-code business application development platform or dynamic case management software may be ideal for creating solutions to most successfully leverage symbiotic RPA and AI.