Joint Salient Object Detection and Existence Prediction

Huaizu Jiang²  Ming-Ming Cheng¹  Shi-Jie Li²  Ali Borji³  Jingdong Wang⁴ 

¹CCCE, Nankai University  ²University of Massachusetts Amherst

    ³Center for Research in Computer Vision, University of Central Florida ⁴Microsoft Research

Abstract

Recent advances in supervised salient object detection modeling has resulted in significant performance improvements on benchmark datasets. However, most of the existing salient object detection models assume that at least one salient object exists in the input image. Such an assumption often leads to less appealing saliency maps on the background images with no salient object at all. Therefore, handling those cases can reduce the false positive rate of a model. In this paper, we propose a supervised learning approach for jointly addressing the salient object detection and existence prediction problems. Given a set of background-only images and images with salient objects, as well as their salient object annotations, we adopt the structural SVM framework and formulate the two problems jointly in a single integrated objective function: saliency labels of superpixels are involved in a classification term conditioned on the salient object existence variable, which in turn depends on both global image and regional saliency features and saliency labels assignments. The loss function also considers both imagelevel and region-level mis-classifications. Extensive evaluation on benchmark datasets validate the effectiveness of our proposed joint approach compared to the baseline and state-of-the-art models. Source code and data is available at http://mmcheng.net/salexist/.

Papers:

• Joint Salient Object Detection and Existence Prediction, Huaizu Jiang, Ming-Ming Cheng, Shi-Jie Li, Ali Borji, Jingdong Wang, Front. Comput. Sci., 2018. [pdf] [bib] [Project Page] [Supplemental] [code]

Most related projects on this website:

• Efficient Salient Region Detection with Soft Image Abstraction. Ming-Ming Cheng, Jonathan Warrell, Wen-Yan Lin, Shuai Zheng, Vibhav Vineet, Nigel Crook. IEEE International Conference on Computer Vision (IEEE ICCV), 2013. [pdf] [Project page] [bib] [latex] [official version]
• BING: Binarized Normed Gradients for Objectness Estimation at 300fp, Ming-Ming Cheng, Ziming Zhang, Wen-Yan Lin, Philip H. S. Torr, IEEE International Conference on Computer Vision and Pattern Recognition (IEEE CVPR), 2014. [Project page][pdf][bib] (Oral, Accept rate: 5.75%)
• SalientShape: Group Saliency in Image Collections. Ming-Ming Cheng, Niloy J. Mitra, Xiaolei Huang, Shi-Min Hu. The Visual Computer 30 (4), 443-453, 2014. [pdf] [Project page] [bib] [latex] [Official version]

Downloads

          SOSB dataset: Weiyun or Google Drive.

Comparisons with state of the art methods

AP, Fθ and MAE scores compared with state-of-the-art approaches on different benchmark datasets, where supervised approaches are marked with bold fonts. The best three scores are highlighted with red, green, and blue fonts, respectively

Precision-Recall curves of different approaches on MSRA-B and ECSSD benchmark datasets

Qualitative comparisons of saliency maps produced by different approaches. From left to right: input images, saliency maps of state-of-the-art approaches, and saliency maps of our proposed approach SSVM.