Nonhuman Primate Brain Tissue Segmentation Using a Transfer Learning Approach

Non - human primates (NHPs) serve as critical models for understanding human brain function and neurological disorders due to their close evolutionary relationship with humans. Accurate brain tissue segmentation in NHPs is critical for understanding neurolog ical disorders, but challenging due to the scarcity of annotated NHP brain MRI datasets, the small size of the NHP brain, the limited resolution of available imaging data and the anatomical differences between human and NHP brains. To address these challen ges, we propose a novel approach utilizing ST U - Net with transfer learning to leverage knowledge transferred from human brain MRI data to enhance segmentation accuracy in the NHP brain MRI, particularly when training data is limited. Specifically, we first train our STU - N et model on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset, allowing our model to learn generalizable features of human brain anatomy. This model is then fine - tuned on a small dataset of vervet brain MRI from The Aging Vervet Colony (AVC) at Wake Forest Alzheimer's Disease Research Center (ADRC) to adapt to the NHP - specific neuroanatomy. This enables accurate segmentation of six key tissue types: grey matter (GM), white matter (WM), CSF, deep grey matter, brainstem, and cerebellum. The combination of STU - N et and transfer learning effectively delineates complex tissue boundaries and captures fine anatomical details specific to NHP brains. Notably, our method demonstrated improvement in segmenting small subcortical structures suc h as putamen and thalamus that are challenging to resolve with limited spatial resolution and tissue contrast, and achieved DSC of over 0.88, IoU over 0.8 and HD95 under 7. This study introduces a robust method for multi - class brain tissue segmentation in NHPs, potentially accelerating research in evolutionary neuroscience and preclinical studies of neurological disorders relevant to human health.