Journal / Conference
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR, 2020)
[PDF link: PENDING]
[Code link: here]
Keywords
Weakly-Supervised Learning, Semantic Segmentation
Abstract
Image-level weakly-supervised semantic segmentation (WSSS) aims at learning semantic segmentation by adopting only image class labels. Existing approaches generally rely on class activation maps (CAM) to generate pseudo-masks and then train segmentation models. The main difficulty is that the CAM estimate only covers partial foreground objects. In this paper, we argue that the critical factor preventing to obtain the full object mask is the classification boundary mismatch problem in applying the CAM to WSSS. Because the CAM is optimized by the classification task, it focuses on the discrimination across different image-level classes. However, the WSSS requires to distinguish pixels sharing the same image-level class to separate them into the foreground and the background. To alleviate this contradiction, we propose an efficient end-to-end Intra-Class Discriminator (ICD) framework, which learns intra-class boundaries to help separate the foreground and the background within each image-level class. Without bells and whistles, our approach achieves the state-of-the- art performance of image label based WSSS, with mIoU 68.0% on the VOC 2012 semantic segmentation benchmark, demonstrating the effectiveness of the proposed approach.
Method/Framework
Learning integral objects with intra-class discriminator for weakly-supervised semantic segmentation. Left: The overall framework of the proposed approach, including a branch for CAM and a branch for the proposed ICD. Right: The framework of our ICD module, which contains a bottom-up estimation branch and a top-down adaptation branch. The final ICD scores for generating seeds are obtained by the adapted predictions.
Experiments
We evaluate the proposed method on the standard Pascal VOC 2012 dataset. Extensive experimental results demonstrate the advantage of employing the intra-class discriminator for weakly-supervised semantic segmentation.
Highlight
• We identify the boundary mismatch problem in applying the CAM to the WSSS, i.e., the gap between image-level inter-class recognition and the desired pixel-level intra-class segmentation.
• We propose an efficient end-to-end Intra-Class Discriminator (ICD) approach to address this problem via learning an intra-class boundary to separate foreground objects and the background.
• We conduct extensive experiments to analyze the effectiveness of our proposed ICD approach. The proposed model achieves the state-of-the-art performance of the image-label based WSSS. Citation
@inproceedings{fan2020icd,
title={Learning Integral Objects with Intra-Class Discriminator for Weakly-Supervised Semantic Segmentation},
author={Fan, Junsong and Zhang, Zhaoxiang and Song, Chunfeng and Tan, Tieniu},
booktitle={Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition},
year={2020}
}
