Multi-scale fully convolutional neural networks for histopathology image segmentation: From nuclear aberrations to the global tissue architecture

Extensive integration of widely different spatial scales, as a “mimicry” of how humans approach analogous tasks, boosts the performance of a standard fully convolutional neural network in cancer segmentation in histopathology images, as shown on three different publicly available datasets. A family of U-Net-based architectures as human operator-inspired multi-scale multi-encoder networks is proposed. The approach can be easily adopted to any encoder-decoder segmentation architecture and extended to multiple path fusions. By use of an additional classification loss, additional encoders for largely different spatial scales as the target scale can be trained in a memory-efficient fashion and with moderate additional cost. Histopathologic diagnosis relies on simultaneous integration of information from a broad range of scales, ranging from nuclear aberrations ( ≈O(0.1μm)) through cellular structures ( ≈O(10μm)) to the global tissue architecture ( ⪆O(1mm)).