HFS: Hierarchical Feature Selection for Efficient Image Segmentation

Ming-Ming Cheng¹        Yun Liu¹     Qibin Hou¹      Jiawang Bian¹      Philip Torr²      Shi-Min Hu³     Zhuowen Tu⁴

¹Nankai University     ²University of Oxford     ³Tsinghua University     ⁴UCSD

Abstract

In this paper, we propose a real-time system, Hierarchical Feature Selection (HFS), that performs image segmentation at a speed of 50 frames-per-second. We make an attempt to improve the performance of previous image segmentation systems by focusing on two aspects: (1) a careful system implementation on modern GPUs for efficient feature computation; and (2) an effective hierarchical feature selection and fusion strategy with learning. Compared with classic segmentation algorithms, our system demonstrates its particular advantage in speed, with comparable results in segmentation quality. Adopting HFS in applications like salient object detection and object proposal generation results in a significant performance boost. Our proposed HFS system (will be open-sourced) can be used in a variety computer vision tasks that are  built on top of image segmentation and superpixel extraction.

Papers

• HFS: Hierarchical Feature Selection for Efficient Image Segmentation. Ming-Ming Cheng, Yun Liu, Qibin Hou, Jiawang Bian, Philip Torr, Shi-Min Hu, Zhuowen Tu, ECCV 2016. [Project page][pdf][bib][C++]

We have released the code and data for plotting the edge PR curves of many existing image segmentation and edge detection methods here.

Hierarchical Feature Selection (HFS)

Fig. 1: Pipeline of our methods.

Fig. 2: HFS for region merging

Evaluation on BSDS500 dataset

Fig. 3: Experimental evaluation for boundaries on BSDS500.

Fig. 4: Experimental evaluation for regions on BSDS500.

Fig. 5: Some examples of EGB, SLIC and our method. Left: source image. Middle left: SLIC [1]. Middle right: EGB [6]. Right: ours.

Links to most related works

[1] Slic superpixels compared to state-of-the-art superpixel methods. Achanta et al. IEEE TPAMI, 2012
[2] Contour detection and hierarchical image segmentation. Arbelaez et al. IEEE TPAMI, 2011
[3] Multiscale combinatorial grouping. Arbelaez et al. IEEE CVPR, 2014
[4] Mean shift: A robust approach toward feature space analysis. Comaniciu et al. IEEE TPAMI, 2002
[5] Spectral segmentation with multiscale graph decomposition. Cour et al. IEEE CVPR, 2005
[6] Ecient graph-based image segmentation. Felzenszwalb et al. IJCV, 2004
[7] Fast and eective l0 gradient minimization by region fusion. Nguyen et al. IEEE ICCV, 2015
[8] gslicr: Slic superpixels at over 250hz. Ren et al. arXiv, 2015
[9] Fast partitioning of vector-valued images. Storath et al. SIAM Journal on Imaging Sciences, 2014