Then we could generally binarize the reduced feature map before fed into the GUST.

Humans are able to effortlessly group the world in terms of abstract and reusable'building blocks',

Neural Information Processing Systems http://nips.cc/

Learning disentangled and distributed representation ofgenerativefactors oftheworld isbelieved tobenefit compositional generalization, because those invariant features canbereused assymbols to build exponentially larger amounts of objects with higher complexity [1, 2, 3, 4].

We address the problem of denoising data from a Gaussian mixture using a two-layer non-linear autoencoder with tied weights and a skip connection.

Polysomnography (PSG), the gold standard test for sleep analysis, generates vast amounts of multimodal clinical data, presenting an opportunity to leverage self-supervised representation learning (SSRL) for pre-training foundation models to enhance sleep analysis. However, progress in sleep foundation models is hindered by two key limitations: (1) the lack of a shared dataset and benchmark with diverse tasks for training and evaluation, and (2) the absence of a systematic evaluation of SSRL approaches across sleep-related tasks. To address these gaps, we introduce Stanford Sleep Bench, a large-scale PSG dataset comprising 17,467 recordings totaling over 163,000 hours from a major sleep clinic, including 13 clinical disease prediction tasks alongside canonical sleep-related tasks such as sleep staging, apnea diagnosis, and age estimation. We systematically evaluate SSRL pre-training methods on Stanford Sleep Bench, assessing downstream performance across four tasks: sleep staging, apnea diagnosis, age estimation, and disease and mortality prediction. Our results show that multiple pretraining methods achieve comparable performance for sleep staging, apnea diagnosis, and age estimation. However, for mortality and disease prediction, contrastive learning significantly outperforms other approaches while also converging faster during pretraining. To facilitate reproducibility and advance sleep research, we will release Stanford Sleep Bench along with pretrained model weights, training pipelines, and evaluation code.

Unsupervised anomaly detection and segmentation methods train a model to learn the training distribution as'normal'. In the testing phase, they identify patterns that deviate from this normal distribution as'anomalies'. To learn the'normal' distribution, prevailing methods corrupt the images and train a model to reconstruct them. During testing, the model attempts to reconstruct corrupted inputs based on the learned'normal' distribution. Deviations from this distribution lead to high reconstruction errors, which indicate potential anomalies. However, corrupting an input image inevitably causes information loss even in normal regions, leading to suboptimal reconstruction and an increased risk of false positives. To alleviate this, we propose IterMask3D, an iterative spatial mask-refining strategy designed for 3D brain MRI. We iteratively spatially mask areas of the image as corruption and reconstruct them, then shrink the mask based on reconstruction error. This process iteratively unmasks'normal' areas to the model, whose information further guides reconstruction of'normal' patterns under the mask to be reconstructed accurately, reducing false positives. In addition, to achieve better reconstruction performance, we also propose using high-frequency image content as additional structural information to guide the reconstruction of the masked area. Extensive experiments on the detection of both synthetic and real-world imaging artifacts, as well as segmentation of various pathological lesions across multiple MRI sequences, consistently demonstrate the effectiveness of our proposed method. Introduction Segmenting anomalies is crucial in the field of medical image analysis as it enables applications such as early disease detection and diagnosis, guides treatment planning, and reduces clinical workload. Conventional anomaly segmentation methods are mostly supervised, relying on annotated training data, where images contain anomalies with corresponding manual labels. Trained on a limited set of data types, the model can only segment anomalies resembling those in its training data, and struggles to detect other types of unseen anomalies. In the context of brain MRI images, the focus of this study, this type of methods typically target a specific pathology (Kamnitsas et al., 2017; Isensee et al., 2021; Stollenga et al., 2015). Unsupervised anomaly segmentation, on the other hand, does not require any'anomalous' images or their manual segmentations during training. Instead, it is trained exclusively on'normal' images and treats this training distribution as the'normal' reference. During testing, the method regards any deviation from this reference as'anomaly' and attempts to segment it.