Results from Randomized Controlled Trials (RCTs) establish the comparative effectiveness of interventions, and are in turn critical inputs for evidence-based care. However, results from RCTs are presented in (often unstructured) natural language articles describing the design, execution, and outcomes of trials; clinicians must manually extract findings pertaining to interventions and outcomes of interest from such articles. This onerous manual process has motivated work on (semi-)automating extraction of structured evidence from trial reports. In this work we propose and evaluate a text-to-text model built on instruction-tuned Large Language Models (LLMs) to jointly extract Interventions, Outcomes, and Comparators (ICO elements) from clinical abstracts, and infer the associated results reported. Manual (expert) and automated evaluations indicate that framing evidence extraction as a conditional generation task and fine-tuning LLMs for this purpose realizes considerable ($\sim$20 point absolute F1 score) gains over the previous SOTA. We perform ablations and error analyses to assess aspects that contribute to model performance, and to highlight potential directions for further improvements. We apply our model to a collection of published RCTs through mid-2022, and release a searchable database of structured findings: http://ico-relations.ebm-nlp.com

Navigating a career constitutes one of life’s most enduring challenges, particularly within a unique organization like the US Navy. While the Navy has numerous resources for guidance, accessing and identifying key information sources across the many existing platforms can be challenging for sailors (e.g., determining the appropriate program or point of contact, developing an accurate understanding of the process, and even recognizing the need for planning itself). Focusing on intermediate goals, evaluations, education, certifications, and training is quite demanding, even before considering their cumulative long-term implications. These are on top of generic personal issues, such as financial difficulties and homesickness when at sea for prolonged periods. We present the preliminary construction of a conversational intelligent agent designed to provide a user-friendly, adaptive environment that recognizes user input pertinent to these issues and provides guidance to appropriate resources within the Navy. User input from “counseling sessions” is linked, using advanced natural language processing techniques, to our framework of Navy training and education standards, promotion protocols, and organizational structure, producing feedback on resources and recommendations sensitive to user history and stated career goals. The proposed innovative technology monitors sailors’ career progress, proactively triggering sessions before major career milestones or when performance drops below Navy expectations, by using a mixed-initiative design. System-triggered sessions involve positive feedback and informative dialogues (using existing Navy career guidance protocols). The intelligent agent also offers counseling for personal problems, triggering targeted dialogues designed to gather more information, offer tailored suggestions, and provide referrals to appropriate resources or to a human counselor when in-depth counseling is warranted. This software, currently in alpha testing, has the potential to serve as a centralized information hub, engaging and encouraging sailors to take ownership of their career paths in the most efficient way possible, benefiting both individuals and the Navy as a whole.

Intelligent learning technologies (ILT) include a diverse array of computer-based systems and tools designed to foster meaningful student learning. These technologies are intelligent to the extent they implement artificial intelligence principles and techniques to create teachable structure from content, analyze and model inputs from the learner, and generate personalized and adaptive feedback and guidance. Intelligent tutoring systems (ITSs) represent a classic example. ITSs, broadly defined, possess an outer loop that intelligently selects the next relevant task, or content object, for learners to complete based on prior performance, and an inner loop that provides iterative and intelligent feedback as learners work toward completing their tasks. However, intelligent learning technologies encompass more than just intelligent tutors. Increasingly, educational games, automated writing evaluation, virtual pedagogical agents, simulations, virtual worlds, open-ended problem solving, generative concept maps, AI-assisted authoring systems, learning content aggregation programs, and e-textbooks rely on some form of artificial intelligence to enrich the learning experience.