DemoResearch

Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton

Jiang-Jiang Liu1 Qibin Hou2 Ming-Ming Cheng1

1CS, Nankai University      2NUS

Online Demo

Abstract

Salient object segmentation, edge detection, and skeleton extraction are three contrasting low-level pixel-wise vision problems, where existing works mostly focused on designing tailored methods for each individual task. However, it is inconvenient and inefficient to store a pre-trained model for each task and perform multiple different tasks in sequence. There are methods that solve specific related tasks jointly but require datasets with different types of annotations supported at the same time. In this paper, we first show some similarities shared by these tasks and then demonstrate how they can be leveraged for developing a unified framework that can be trained end-to-end. In particular, we introduce a selective integration module that allows each task to dynamically choose features at different levels from the shared backbone based on its own characteristics. Furthermore, we design a task-adaptive attention module, aiming at intelligently allocating information for different tasks according to the image content priors. To evaluate the performance of our proposed network on these tasks, we conduct exhaustive experiments on multiple representative datasets. We will show that though these tasks are naturally quite different, our network can work well on all of them and even perform better than current single-purpose state-of-the-art methods. In addition, we also conduct adequate ablation analyses that provide a full understanding of the design principles of the proposed framework.

Paper

  • Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton, Jiang-Jiang Liu, Qibin Hou, Ming-Ming Cheng. IEEE TIP, 2020.  [pdf][code]

If you find our work is helpful, please cite

@article{liu2020dynamic,
title={Dynamic Feature Integration for Simultaneous Detection of Salient Object, Edge and Skeleton},
author={Jiang-Jiang Liu and Qibin Hou and Ming-Ming Cheng},
journal={IEEE Transactions on Image Processing},
year={2020},
volume={},
number={},
pages={1-15},
doi={10.1109/TIP.2020.3017352},
}
@inproceedings{Liu19PoolNet,
  title={A Simple Pooling-Based Design for Real-Time Salient Object Detection},
  author={Jiang-Jiang Liu and Hou, Qibin and Cheng, Ming-Ming and Feng, Jiashi and Jiang, Jianmin},
  booktitle={IEEE Conference on Computer Vision and Pattern Recognition},
  year={2019},
}

Contact

j04.liu AT gmail DOT com

(Visited 3,842 times, 1 visits today)
Subscribe
Notify of
guest

2 Comments
Inline Feedbacks
View all comments
qing

It is very interesting and significant that utilize diverse datasets,but i do not understand different type image and lable ,how this network structure can tell the better optimize direction,just like tiger,cat dog,totally different.

MM Cheng

Each type of annotation is used to update its corresponding detection head (small amount of parameters) + the shared part (most amount of data).