electron microscopy
Contrast transfer functions help quantify neural network out-of-distribution generalization in HRTEM
DaCosta, Luis Rangel, Scott, Mary C.
Neural networks, while effective for tackling many challengi ng scientific tasks, are not known to perform well out-of-distribution (OOD), i.e., within domains which d iffer from their training data. Understanding neural network OOD generalization is paramount to their suc cessful deployment in experimental workflows, especially when ground-truth knowledge about the experime nt is hard to establish or experimental conditions significantly vary. With inherent access to ground-truth in formation and fine-grained control of underlying distributions, simulation-based data curation facilitate s precise investigation of OOD generalization behavior. Here, we probe generalization with respect to imaging condi tions of neural network segmentation models for high-resolution transmission electron microscopy (HRTEM) imaging of nanoparticles, training and measuring the OOD generalization of over 12,000 neural networks using synthetic data generated via random structure sampling and multislice simulation. Using the HRTEM contra st transfer function, we further develop a framework to compare information content of HRTEM datasets an d quantify OOD domain shifts. We demonstrate that neural network segmentation models enjoy significant performance stability, but will smoothly and predictably worsen as imaging conditions shift from the training distribution. Lastly, we consider limitations of our approach in explaining other OOD shifts, s uch as of the atomic structures, and discuss complementary techniques for understanding generalizatio n in such settings.
FlowTIE: Flow-based Transport of Intensity Equation for Phase Gradient Estimation from 4D-STEM Data
Bangun, Arya, Tรถllner, Maximilian, Zhao, Xuan, Kรผbel, Christian, Scharr, Hanno
We introduce FlowTIE, a neural-network-based framework for phase reconstruction from 4D-Scanning Transmission Electron Microscopy (STEM) data, which integrates the Transport of Intensity Equation (TIE) with a flow-based representation of the phase gradient. This formulation allows the model to bridge data-driven learning with physics-based priors, improving robustness under dynamical scattering conditions for thick specimen. The validation on simulated datasets of crystalline materials, benchmarking to classical TIE and gradient-based optimization methods are presented. The results demonstrate that FlowTIE improves phase reconstruction accuracy, fast, and can be integrated with a thick specimen model, namely multislice method.
Revealing the Hidden Third Dimension of Point Defects in Two-Dimensional MXenes
Guinan, Grace, Smeaton, Michelle A., Wyatt, Brian C., Goldy, Steven, Egan, Hilary, Glaws, Andrew, Tucker, Garritt J., Anasori, Babak, Spurgeon, Steven R.
Point defects govern many important functional properties of two - dimensional ( 2D) materials. However, resolving the three - dimensional (3D) arrangement of these defects in multi - layer 2D materials remains a fundamental challenge, hindering rational defect engineering . Our approach reconstructs the 3D coordinates of vacancies across hundreds of thousands of lattice sites, generating robust statistical insight into their dist ribution that can be correlated with specinullic synthesis pathways. This large - scale data enables us to classify a hierarchy of defect structures -- from isolated vacancies to nanopores -- revealing their preferred formation and interaction mechanisms, as corroborated by molecular dynamics simulations . This work provides a generalizable framework for understanding and ultimately controlling point defects across large volumes, paving the way for the rational design of defect - engineered functional 2D materials. Keywords: 2D materials, point defects, autonomous materials science, electron microscopy, machine learning 2 Two - dimensional (2D) materials have become a major nullield of modern research in materials science after the discovery of graphene in 2004 . The challenge of characterizing point defects is signinullicantly amplinullied in few - layered 2D materials. For instance, MXenes -- a class of 2D transition metal carbides, carbonitrides, and nitrides -- consist of nanosheets containing two to nullive layers of metal ato ms, which complicates defect analysis compared to single - layer materials .
gACSON software for automated segmentation and morphology analyses of myelinated axons in 3D electron microscopy
Behanova, Andrea, Abdollahzadeh, Ali, Belevich, Ilya, Jokitalo, Eija, Sierra, Alejandra, Tohka, Jussi
Background and Objective: Advances in electron microscopy (EM) now allow three-dimensional (3D) imaging of hundreds of micrometers of tissue with nanometer-scale resolution, providing new opportunities to study the ultra-structure of the brain. In this work, we introduce a freely available Matlab-based gACSON software for visualization, segmentation, assessment, and morphology analysis of myelinated axons in 3D-EM volumes of brain tissue samples. Methods: The software is equipped with a graphical user interface (GUI). It automatically segments the intra-axonal space of myelinated axons and their corresponding myelin sheaths and allows manual segmentation, proofreading, and interactive correction of the segmented components. Results: We illustrate the use of the software by segmenting and analyzing myelinated axons in six 3D-EM volumes of rat somatosensory cortex after sham surgery or traumatic brain injury (TBI). Our results suggest that the equivalent diameter of myelinated axons in somatosensory cortex was decreased in TBI animals five months after the injury. Conclusions: Our results indicate that gACSON is a valuable tool for visualization, segmentation, assessment, and morphology analysis of myelinated axons in 3D-EM volumes. Introduction Assessing the structure of the brain is critical to better understanding its normal and abnormal functioning. Advances in electron microscopy (EM) now allow three-dimensional (3D) imaging of hundreds of micrometers of tissue with nanometer-scale resolution, providing new opportunities to study the ultrastructure of the brain [1, 2]. Quantitative analysis of 3D-EM data, such as morphological assessment of ultrastructure, spatial distribution or connectivity of cells, requires the instance segmentation of individual ultrastructural components [3, 4, 5]. Performing this segmentation manually is tedious, if not impossible, due to the large size and enormous number of components in typical 3D-EM data.
Digitizing Spermatogenesis Lineage at Nanoscale Resolution In Tissue-Level Electron Microscopy
Xiao, Li, Liu, Liqing, Wu, Hongjun, Zhong, Jiayi, Zhang, Yan, Hu, Junjie, Fei, Sun, Yang, Ge, Xu, Tao
School of Life Sciences, University of Chinese Academy of Sciences, Beijing 100049, China # These authors contributed equally to this work. Email: andrewxiao@bupt.edu.cn;liuliqing@ibp.ac.cn;huj@ibp.ac.cn; feisun@ibp.ac.cn; yangge@ucas.edu.cn;xutao@ibp.ac.cn ABSTRACT Recent advances in 2D large - scale and 3D volume electron microscopy have stimulated the rapid development of nanoscale functional analysis at the tissue and organ levels. To meet the requirements of characterizing intracellular organelle s and their interactions within defined cellular cohorts at tissue level, we have developed DeepOrganelle. It adopts a lightweighted Mask2Former frameworks as a universal segmentor and is capable of segmenting and extracting organelles within different cell types, performing statistical quantitative analysis, as well as visualizing and quantifying the spatial distribution of organelle morphologies and interactions across different cell types at tissue scales. Using DeepOrganelle, we systemically perform cross - scale quantification of membrane contact sites( MCSs) dynamics across the progression of the seminiferous epithelial cycle, covering 12 distinct developmental stages and 24 statuses of germ cells . Noticeably, it discovers a waved pattern of mitochondria(Mito) - endoplasmic reticulum(ER) contact with a significant increase specifically at Stage X pachytene preceding the transition to diplotene, which aligns well with a newly reported experiment that mitochondrial metabolic proteins like PDHA2 are essential for this transition by maintaining ATP supply for double - strand break (DSB) repair.
Can Multimodal LLMs See Materials Clearly? A Multimodal Benchmark on Materials Characterization
Lai, Zhengzhao, Zheng, Youbin, Cai, Zhenyang, Lyu, Haonan, Yang, Jinpu, Liang, Hongqing, Hu, Yan, Wang, Benyou
Materials characterization is fundamental to acquiring materials information, revealing the processing-microstructure-property relationships that guide material design and optimization. While multimodal large language models (MLLMs) have recently shown promise in generative and predictive tasks within materials science, their capacity to understand real-world characterization imaging data remains underexplored. To bridge this gap, we present MatCha, the first benchmark for materials characterization image understanding, comprising 1,500 questions that demand expert-level domain expertise. MatCha encompasses four key stages of materials research comprising 21 distinct tasks, each designed to reflect authentic challenges faced by materials scientists. Our evaluation of state-of-the-art MLLMs on MatCha reveals a significant performance gap compared to human experts. These models exhibit degradation when addressing questions requiring higher-level expertise and sophisticated visual perception. Simple few-shot and chain-of-thought prompting struggle to alleviate these limitations. These findings highlight that existing MLLMs still exhibit limited adaptability to real-world materials characterization scenarios. We hope MatCha will facilitate future research in areas such as new material discovery and autonomous scientific agents. MatCha is available at https://github.com/FreedomIntelligence/MatCha.
Review of Deep Learning Applications to Structural Proteomics Enabled by Cryogenic Electron Microscopy and Tomography
Zhou, Brady K., Hu, Jason J., Lee, Jane K. J., Zhou, Z. Hong, Terzopoulos, Demetri
The past decade has witnessed a transformative "cryoEM revolution" characterized by exponential growth in high - resolution structural data, driven by advances in cryogenic electron microscopy (cryoEM) and cryogenic electron t omography (cryoET). The integration of deep learning technologies into structural proteomics workflows has emerged as a pivotal force in addressing longstanding challenges, including low signal - to - noise ratios, preferred orientation artifacts, and missing - wedge problems th at have historically limited efficiency and scalability. This review article examines the application of Artificial Intelligence (AI) across the entire cryoEM pipeline, from automated particle picking using convolutional neural networks (Topaz, crYOLO, Cry oSegNet) to computational solutions for preferred orientation bias (spIsoNet, cryoPROS) and advanced denoising algorithms (Topaz - Denoise). In cryoET, tools such as IsoNet employ U - Net architectures for simultaneous missing - wedge correction and noise reduct ion, while TomoNet streamlines subtomogram averaging through AI - driven particle detection. The workflow culminates with automated atomic model building using sophisticated tools like ModelAngelo, DeepTracer, and CryoREAD that translate density maps into in terpretable biological structures. These AI - enhanced approaches have demonstrated remarkable achievements, including near - atomic resolution reconstructions with minimal manual intervention, resolution of previously intractable datasets suffering from sever e orientation bias, and successful application to diverse biological systems from HIV virus - like particles to in situ ribosomal complexes. As deep learning continues to evolve, particularly with the emergence of large language models and vision transformer s, the future promises even more sophisticated automation and accessibility in structural biology, potentially revolutionizing our understanding of macromolecular architecture and function.