A 2017 Guide to Semantic Segmentation with Deep Learning


At Qure, we regularly work on segmentation and object detection problems and we were therefore interested in reviewing the current state of the art. In this post, I review the literature on semantic segmentation. Although the results are not directly applicable to medical images, I review these papers because research on the natural images is much more mature than that of medical images. Post is organized as follows: I first explain the semantic segmentation problem, give an overview of the approaches and summarize a few interesting papers. In a later post, I'll explain why medical images are different from natural images and examine how the approaches from this review fare on a dataset representative of medical images.

Design of Real-time Semantic Segmentation Decoder for Automated Driving

arXiv.org Machine Learning

Semantic segmentation remains a computationally intensive algorithm for embedded deployment even with the rapid growth of computation power. Thus efficient network design is a critical aspect especially for applications like automated driving which requires real-time performance. Recently, there has been a lot of research on designing efficient encoders that are mostly task agnostic. Unlike image classification and bounding box object detection tasks, decoders are computationally expensive as well for semantic segmentation task. In this work, we focus on efficient design of the segmentation decoder and assume that an efficient encoder is already designed to provide shared features for a multi-task learning system. We design a novel efficient non-bottleneck layer and a family of decoders which fit into a small run-time budget using VGG10 as efficient encoder. We demonstrate in our dataset that experimentation with various design choices led to an improvement of 10\% from a baseline performance.

Diving into Deep Convolutional Semantic Segmentation Networks and Deeplab_V3


Deep Convolutional Neural Networks (DCNNs) have achieved remarkable success in various Computer Vision applications. Like others, the task of semantic segmentation is not an exception to this trend. This piece provides an introduction to Semantic Segmentation with a hands-on TensorFlow implementation. Regular image classification DCNNs have similar structure. These models take images as input and output a single value representing the category of that image. Usually, classification DCNNs have four main operations. Note that in this setup, we categorize an image as a whole.

DSNet for Real-Time Driving Scene Semantic Segmentation

arXiv.org Artificial Intelligence

We focus on the very challenging task of semantic segmentation for autonomous driving system. It must deliver decent semantic segmentation result for traffic critical objects real-time. In this paper, we propose a very efficient yet powerful deep neural network for driving scene semantic segmentation termed as Driving Segmentation Network (DSNet). DSNet achieves state-of-the-art balance between accuracy and inference speed through efficient units and architecture design inspired by ShuffleNet V2 and ENet. More importantly, DSNet highlights classes most critical with driving decision making through our novel Driving Importance-weighted Loss. We evaluate DSNet on Cityscapes dataset, our DSNet achieves 71.8% mean Intersection-over-Union (IoU) on validation set and 69.3% on test set. Class-wise IoU scores show that Driving Importance-weighted Loss could improve most driving critical classes by a large margin. Compared with ENet, DSNet is 18.9% more accurate and 1.1+ times faster which implies great potential for autonomous driving application.

Trace-back Along Capsules and Its Application on Semantic Segmentation

arXiv.org Machine Learning

In this paper, we propose a capsule-based neural network model to solve the semantic segmentation problem. By taking advantage of the extractable part-whole dependencies available in capsule layers, we derive the probabilities of the class labels for individual capsules through a recursive, layer-by-layer procedure. We model this procedure as a traceback pipeline and take it as a central piece to build an end-to-end segmentation network. Under the proposed framework, image-level class labels and object boundaries are jointly sought in an explicit manner, which poses a significant advantage over the state-of-the-art fully convolutional network (FCN) solutions. Experiments conducted on modified MNIST and neuroimages demonstrate that our model considerably enhance the segmentation performance compared to the leading FCN variant.