Global2Local: Efficient Structure Search for Video Action Segmentation

04/01/2021 Shang-Hua Gao

IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2021

Shang-Hua Gao#, Qi Han#, Zhong-Yu Li, Pai Peng, Liang Wang, Ming-Ming Cheng

# indicates joint first authors

Introduction

Temporal receptive fields of models play an important role in action segmentation. Large receptive fields facilitate the long-term relations among video clips while small receptive fields help capture the local details. Existing methods construct models with hand-designed receptive fields in layers. Can we effectively search for receptive field combinations to replace hand-designed patterns? To answer this question, we propose to find better receptive field combinations through a global-to-local search scheme. Our search scheme exploits both global search to find the coarse combinations and local search to get the refined receptive field combination patterns further. The global search finds possible coarse combinations other than human-designed patterns. On top of the global search, we propose an expectation guided iterative local search scheme to refine combinations effectively. Our global-to-local search can be plugged into existing action segmentation methods to achieve state-of-the-art performance.

Codes

Source Code and pre-trained model: https://github.com/ShangHua-Gao/G2L-search

Paper Links

Global2Local: Efficient Structure Search for Video Action Segmentation, Shang-Hua Gao#, Qi Han#, Zhong-Yu Li, Pai Peng, Liang Wang, Ming-Ming Cheng*, IEEE CVPR, 2021. [pdf|code|bib]

Citation

@inproceedings{gao2021global2local,   
title={Global2Local: Efficient Structure Search for Video Action Segmentation},   
author={Gao, Shang-Hua and Han, Qi and Li, Zhong-Yu and Peng, Pai and Wang, Liang and Cheng, Ming-Ming},   
booktitle=CVPR,   
year={2021} }