Deep Convolutional Neural Networks with Spatial Regularization, Volume and Star-Shape Priors for Image Segmentation

Deep Convolutional Neural Networks (DCNNs) can well extract the features from natural images. However, the classification functions in the existing network architecture of CNNs are simple and lack capabilities to handle important spatial information as have been done by many well-known traditional variational image segmentation models. Priors such as spatial regularization, volume prior and shapes priors cannot be handled by existing DCNNs. We propose a novel Soft Threshold Dynamics (STD) framework which can integrate many spatial priors of the classic variational models into the DCNNs for image segmentation. The novelty of our method is to interpret the softmax activation function as a dual variable in a variational problem, and thus many spatial priors can be imposed in the dual space. From this viewpoint, we can build a STD based framework which can enable the outputs of DCNNs to have many special priors such as spatial regularization, volume preservation and star-shape prior. The proposed method is a general mathematical framework and it can be applied to any image segmentation DCNNs with a softmax classification layer. To show the efficiency of our method, we applied it to the popular DeepLabV3+ image segmentation network, and the experiments results show that our method can work efficiently on data-driven image segmentation DCNNs.