Recognition from Web Data: A Progressive Filtering Approach

Abstract

Leveraging the abundant number of web data is a promising strategy in addressing the problem of data lacking when training convolutional neural networks (CNNs). However, web images often contain incorrect tags, which may compromise the learned CNN model. To address this problem, this paper focuses on image classification and proposes to iterate between filtering out noisy web labels and fine-tuning the CNN model using the crawled web images. Overall, the proposed method benefits from the growing modeling capability of the learned model to correct labels for web images and learning from such new data to produce a more effective model. Our contribution is two-fold. First, we propose an iterative method that progressively improves the discriminative ability of CNNs and the accuracy of web image selection. This method is beneficial towards selecting high-quality web training images and expanding the training set as the model gets ameliorated. Second, since web images are usually complex and may not be accurately described by a single tag, we propose to assign a web image multiple labels to reduce the impact of hard label assignment. This labeling strategy mines more training samples to improve the CNN model. In the experiments, we crawl 0.5 million web images covering all categories of four public image classification datasets. Compared with the baseline which has no web images for training, we show that the proposed method brings notable improvement. We also report competitive recognition accuracy compared with the state of the art.

Paper

Recognition from web data: A progressive filtering approach，Jufeng Yang, Xiaoxiao Sun, Yu-Kun Lai, Liang Zheng, Ming-Ming Cheng. IEEE Transactions on Image Processing (TIP), 27(11): 5303-5315, 2019. [pdf] [project page]

Method

Pipeline of the proposed progressive learning method, which includes two major steps, namely data sampling and model updating. For data sampling, we obtain P(y|x), score matrix S and label matrix R of web data based on model M_t−1, and use such information to obtain the dataset Det−1 through a sampling scheme. Here, one to one and one to many are two types of label sampling strategies. For model updating, model M_t, is initialized with the parameters of M_t−1 and updated on the combined dataset in the t-th iteration.

Performance

Accuracies on four datasets shows that the proposed method outperforms other methods learning from web data. Here, Baseline+A (indicates that progressive filtering is employed), and Baseline+B shows one-to-many correction is used (instead of one-to-one correction). “∗” is obtained by employing the modification of domain adaptation model [52] to further reduce the effect of different data distribution.