light microscopy
BIOMEDICA: An Open Biomedical Image-Caption Archive, Dataset, and Vision-Language Models Derived from Scientific Literature
Lozano, Alejandro, Sun, Min Woo, Burgess, James, Chen, Liangyu, Nirschl, Jeffrey J, Gu, Jeffrey, Lopez, Ivan, Aklilu, Josiah, Katzer, Austin Wolfgang, Chiu, Collin, Rau, Anita, Wang, Xiaohan, Zhang, Yuhui, Song, Alfred Seunghoon, Tibshirani, Robert, Yeung-Levy, Serena
The development of vision-language models (VLMs) is driven by large-scale and diverse multimodal datasets. However, progress toward generalist biomedical VLMs is limited by the lack of annotated, publicly accessible datasets across biology and medicine. Existing efforts are restricted to narrow domains, missing the full diversity of biomedical knowledge encoded in scientific literature. To address this gap, we introduce BIOMEDICA, a scalable, open-source framework to extract, annotate, and serialize the entirety of the PubMed Central Open Access subset into an easy-to-use, publicly accessible dataset. Our framework produces a comprehensive archive with over 24 million unique image-text pairs from over 6 million articles. Metadata and expert-guided annotations are also provided. We demonstrate the utility and accessibility of our resource by releasing BMCA-CLIP, a suite of CLIP-style models continuously pre-trained on the BIOMEDICA dataset via streaming, eliminating the need to download 27 TB of data locally. On average, our models achieve state-of-the-art performance across 40 tasks - spanning pathology, radiology, ophthalmology, dermatology, surgery, molecular biology, parasitology, and cell biology - excelling in zero-shot classification with a 6.56% average improvement (as high as 29.8% and 17.5% in dermatology and ophthalmology, respectively), and stronger image-text retrieval, all while using 10x less compute. To foster reproducibility and collaboration, we release our codebase and dataset for the broader research community.
- North America > United States > California > Santa Clara County > Palo Alto (0.05)
- South America > Chile > Santiago Metropolitan Region > Santiago Province > Santiago (0.04)
- North America > United States > Maryland > Montgomery County > Bethesda (0.04)
- (3 more...)
- Health & Medicine > Therapeutic Area > Infections and Infectious Diseases (1.00)
- Health & Medicine > Diagnostic Medicine > Imaging (1.00)
- Health & Medicine > Therapeutic Area > Neurology (0.92)
- Health & Medicine > Therapeutic Area > Oncology > Carcinoma (0.46)
- Information Technology > Artificial Intelligence > Vision (1.00)
- Information Technology > Artificial Intelligence > Natural Language > Large Language Model (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Statistical Learning (1.00)
- Information Technology > Artificial Intelligence > Machine Learning > Neural Networks > Deep Learning (0.93)
AI-based automated active learning for discovery of hidden dynamic processes: A use case in light microscopy
Friederich, Nils, Sitcheu, Angelo Yamachui, Neumann, Oliver, Eroğlu-Kayıkçı, Süheyla, Prizak, Roshan, Hilbert, Lennart, Mikut, Ralf
In the biomedical environment, experiments assessing dynamic processes are primarily performed by a human acquisition supervisor. Contemporary implementations of such experiments frequently aim to acquire a maximum number of relevant events from sometimes several hundred parallel, non-synchronous processes. Since in some high-throughput experiments, only one or a few instances of a given process can be observed simultaneously, a strategy for planning and executing an efficient acquisition paradigm is essential. To address this problem, we present two new methods in this paper. The first method, Encoded Dynamic Process (EDP), is Artificial Intelligence (AI)-based and represents dynamic processes so as to allow prediction of pseudo-time values from single still images. Second, with Experiment Automation Pipeline for Dynamic Processes (EAPDP), we present a Machine Learning Operations (MLOps)-based pipeline that uses the extracted knowledge from EDP to efficiently schedule acquisition in biomedical experiments for dynamic processes in practice. In a first experiment, we show that the pre-trained State-Of-The- Art (SOTA) object segmentation method Contour Proposal Networks (CPN) works reliably as a module of EAPDP to extract the relevant object for EDP from the acquired three-dimensional image stack.
High-tech microscope with ML software for detecting malaria in returning travellers
Malaria is an infectious disease claiming more than half a million lives each year. Because traditional diagnosis takes expertise and the workload is high, an international team of researchers investigated if diagnosis using a new system combining an automatic scanning microscope and AI is feasible in clinical settings. They found that the system identified malaria parasites almost as accurately as experts staffing microscopes used in standard diagnostic procedures. This may help reduce the burden on microscopists and increase the feasible patient load. Each year, more than 200 million people fall sick with malaria and more than half a million of these infections lead to death. The World Health Organization recommends parasite-based diagnosis before starting treatment for the disease caused by Plasmodium parasites.
Neural network method for enhancing electron microscope images
Since the early 1930s, electron microscopy has provided unprecedented access to the world of the extraordinarily small, revealing intricate details that are otherwise impossible to discern with conventional light microscopy. But to achieve high resolution over a large sample area, the energy of the electron beams needs to be cranked up, which is costly and detrimental to the sample under observation. Texas A&M University researchers may have found a new method to improve the quality of low-resolution electron micrographs without compromising the integrity of samples. By training deep neural networks on pairs of images from the same sample but at different physical resolutions, they have found that details in lower-resolution images can be enhanced further. "Normally, a high-energy electron beam is passed through the sample at locations where greater image resolution is desired. But with our image processing techniques, we can super-resolve an entire image by using just a few smaller-sized, high-resolution images," said Yu Ding, Professor in the Department of Industrial and Systems Engineering.
- North America > United States > Texas (0.28)
- North America > United States > Virginia (0.05)
Superhuman "cell-sight" with Deep Learning – Towards Data Science
An analysis of the paper In Silico Labeling: Predicting Fluorescent Labels in Unlabeled Images published in Cell. Take a look at this image, and tell me what you see. If you're a decently smart human being with some background in biology (or not), you probably guessed "some cells?" To be more specific, this a human motor neuron culture derived from induced pluripotent stem cells. If you somehow managed to guess that, let me ask you some more questions.