We refer to the proposed solution as SA3D, for Segment Anything in 3D. It is only required to provide a manual segmentation prompt ( e.g., rough points) for the target object in a single view, which is used to generate its 2D mask in this view with SAM.

We refer to the proposed solution as SA3D, for Segment Anything in 3D. It is only required to provide a manual segmentation prompt ( e.g., rough points) for the target object in a single view, which is used to generate its 2D mask in this view with SAM.

Abstract--Reconstructing building floor plans from point cloud data is a critical technology for indoor navigation, building information modeling (BIM), and highly accurate precise indoor measurement applications. Traditional methods, such as geometric algorithms and Mask R-CNN-based deep learning for mask segmentation, often suffer from sensitivity to noise, limited generalization, and loss of geometric details, severely impacting measurement accuracy. This study proposes an innovative framework, FloorSAM, that integrates room-height point cloud density maps with the guided segmentation capabilities of the Segment Anything Model (SAM) to enhance the precision of floor plan reconstruction from LiDAR point cloud data. By applying grid-based filtering to retain elevation point clouds near the ceiling of each region, combined with adaptive resolution projection and image enhancement techniques, a top-down density map is generated, improving the robustness and accuracy of spatial feature measurement. This framework leverages SAM's zero-shot learning to achieve high-fidelity room segmentation, remarkably enhancing reconstruction and measurement accuracy across diverse building layouts. Subsequently, leveraging SAM's zero-shot guided segmentation capabilities, high-quality room masks are generated based on adaptive prompt points, followed by a multistage filtering process to extract precise semantic masks for individual rooms. Through joint analysis of mask and point cloud modalities, contour extraction and regularization are performed, integrating semantic segmentation with geometric information to produce accurate room floor plans and recover topological relationships between rooms.

Recent progress in interactive point prompt based Image Segmentation allows to significantly reduce the manual effort to obtain high quality semantic labels. State-of-the-art unsupervised methods use self-supervised pre-trained models to obtain pseudo-labels which are used in training a prompt-based segmentation model. In this paper, we propose a novel unsupervised and training-free approach based solely on the self-attention of Stable Diffusion. We interpret the self-attention tensor as a Markov transition operator, which enables us to iteratively construct a Markov chain. Pixel-wise counting of the required number of iterations along the Markov-chain to reach a relative probability threshold yields a Markov-iteration-map, which we simply call a Markov-map. Compared to the raw attention maps, we show that our proposed Markov-map has less noise, sharper semantic boundaries and more uniform values within semantically similar regions. We integrate the Markov-map in a simple yet effective truncated nearest neighbor framework to obtain interactive point prompt based segmentation. Despite being training-free, we experimentally show that our approach yields excellent results in terms of Number of Clicks (NoC), even outperforming state-of-the-art training based unsupervised methods in most of the datasets.

The high-throughput quickly changing of the grains, but rule-based segmentation models fail due research environment makes zero-shot generalization an to weak grain boundary contrast and contrast fluctuations incredibly desirable feature. For this reason, we explore across the image. Previous work has gotten around this the application of Meta's Segment Anything Model (SAM) problem by training neural networks on hundreds of handlabeled to this problem. We first analyze the out-of-the-box use of grains [12, 18]. This method works well for segmenting SAM. Then we discuss opportunities and strategies for improvement grain images if the material and image conditions under the assumption of minimal labeled data are not changing, but in the fast-paced semiconductor availability. Out-of-the-box SAM shows promising accuracy research environment, processing conditions and material at property distribution extraction. We are able to identify composition may be changing on even a wafer-to-wafer basis.

Despite the remarkable success of deep learning in medical imaging analysis, medical image segmentation remains challenging due to the scarcity of high-quality labeled images for supervision. Further, the significant domain gap between natural and medical images in general and ultrasound images in particular hinders fine-tuning models trained on natural images to the task at hand. In this work, we address the performance degradation of segmentation models in low-data regimes and propose a prompt-less segmentation method harnessing the ability of segmentation foundation models to segment abstract shapes. We do that via our novel prompt point generation algorithm which uses coarse semantic segmentation masks as input and a zero-shot prompt-able foundation model as an optimization target. We demonstrate our method on a segmentation findings task (pathologic anomalies) in ultrasound images. Our method's advantages are brought to light in varying degrees of low-data regime experiments on a small-scale musculoskeletal ultrasound images dataset, yielding a larger performance gain as the training set size decreases.

Training segmentation models for medical images continues to be challenging due to the limited availability of data annotations. Segment Anything Model (SAM) is a foundation model that is intended to segment user-defined objects of interest in an interactive manner. While the performance on natural images is impressive, medical image domains pose their own set of challenges. Here, we perform an extensive evaluation of SAM's ability to segment medical images on a collection of 19 medical imaging datasets from various modalities and anatomies. We report the following findings: (1) SAM's performance based on single prompts highly varies depending on the dataset and the task, from IoU=0.1135 for spine MRI to IoU=0.8650 for hip X-ray. (2) Segmentation performance appears to be better for well-circumscribed objects with prompts with less ambiguity and poorer in various other scenarios such as the segmentation of brain tumors. (3) SAM performs notably better with box prompts than with point prompts. (4) SAM outperforms similar methods RITM, SimpleClick, and FocalClick in almost all single-point prompt settings. (5) When multiple-point prompts are provided iteratively, SAM's performance generally improves only slightly while other methods' performance improves to the level that surpasses SAM's point-based performance. We also provide several illustrations for SAM's performance on all tested datasets, iterative segmentation, and SAM's behavior given prompt ambiguity. We conclude that SAM shows impressive zero-shot segmentation performance for certain medical imaging datasets, but moderate to poor performance for others. SAM has the potential to make a significant impact in automated medical image segmentation in medical imaging, but appropriate care needs to be applied when using it.