Self-paced Balance Learning for Clinical Skin Disease Recognition

Jufeng Yang¹, Xiaoping Wu¹, Jie Liang¹, Xiaoxiao Sun¹, Ming-Ming Cheng¹, Paul L. Rosin², and Liang Wang³

¹Nankai University ²Cardiff University ³Institute of Automation, CAS

Abstract

Class imbalance is a challenging problem in many classification tasks. It induces biased classification results for minority classes which contain less training samples than others. Most existing approaches aim to remedy the imbalanced number of instances among categories by re-sampling the majority and minority classes accordingly. However, the imbalanced level of difficulty of recognizing different categories is also crucial, especially for distinguishing samples with many classes. For example, in the task of clinical skin disease recognition, several rare diseases have a small number of training samples, but they are easy to diagnose because of their distinct visual properties. On the other hand, some common skin diseases, e.g., eczema, are hard to recognize due to the lack of special symptoms. To address this problem, we propose a self-paced balance learning (SPBL) algorithm in this paper. Specifically, we introduce a comprehensive metric termed the complexity of image category which is a combination of both sample number and recognition difficulty. First, the complexity is initialized using the model of the first pace, where the pace indicates one iteration in the self-paced learning paradigm. We then assign each class a penalty weight which is larger for more complex categories and smaller for easier ones, after which the curriculum is reconstructed by rearranging the training samples. Consequently, the model can iteratively learn discriminative representations via balancing the complexity in each pace. Experimental results on the SD-198 and SD-260 benchmark datasets demonstrate that the proposed SPBL algorithm performs favorably against the state-of-the-art methods. We also demonstrate the effectiveness of the SPBL algorithm’s generalization capacity on various tasks such as indoor scene image recognition, object classification, etc.

Paper

Self-paced Balance Learning for Clinical Skin Disease Recognition. Jufeng Yang, Xiaoping Wu, Jie Liang, Xiaoxiao Sun, Ming-Ming Cheng, Paul L. Rosin, and Liang Wang. TNNLS, 2019. [pdf] [official page] [Project Page & Dataset]

Motivation

The number of training samples for each skin disease depends heavily on its incidence [1]–[3]. Actually, there are more than 1000 kinds of skin diseases, both common and uncommon, for which it is difficult to either collect or annotate a balanced data set. Fig. 1 shows the histograms of image number distributions in two skin disease data sets, i.e., SD-198 (top) [4] and SD-260 (bottom), where the images are captured by the digital camera or mobile phone, uploaded by patients, and labeled by doctor volunteers. In Fig. 1, the blue bars reflect a large gap in the number of samples between common skin diseases, e.g., solar elastosis (SE), allergic contact dermatitis (ACD), acne vulgaris (AV), and benign keratosis (BK), and uncommon skin diseases, e.g., vitiligo (VI), stomatitis (ST), pilomatrixoma (PI), and histiocytosis X (HX). However, as shown by the red line, the recognition accuracy of each category is independent of the number of samples, indicating that the recognition difficulty is also imbalanced for the disease classes. According to empirical medical knowledge [5], some rare skin diseases, e.g., ST and HX, have distinct characteristics and are easy to diagnose, while some common skin diseases, e.g., ACD and BK, are difficult to recognize due to the lack of special symptoms.

Figure 1. Visualization of the class distribution in the SD-198 (top) and SD-260 (bottom) datasets. The blue bars denote the number of training samples (Num) for each class, while the red line denotes the classification accuracy (Acc) of the raw ResNet50 on the testing set. Each colored box visualizes a specific skin disease, e.g., solar elastosis (SE), allergic contact dermatitis (ACD). The numbers in the boxes report the number of samples and the recognition accuracy, respectively.

Method

In this paper, we address the class imbalance problem via a combined complexity metric termed the complexity of image category which synthesizes both the sample number and recognition difficulty of classes. We then design a self-paced balance learning (SPBL) framework inspired by the self-paced learning (SPL) paradigm to construct a dynamic program according to the updated complexity. Here, the SPL simulates the process of teaching a curriculum for students which arranges the samples from easy to difficult during training. It guides the learning procedure to avoid biased results towards the easily recognized categories (e.g., those with large class sizes and small intra-class variation). In addition, we use the iterative SPL scheme to arrange the learning process using the complexity of image categories.

Figure 2. Main steps of the proposed SPBL algorithm. It iteratively trains the weighted SVM classifier and updates the self-paced curriculum. The predictions with top scores form the initial curriculum Φ. During training, the algorithm calculates the distribution of class complexity level H, which combines both the class size and recognition difficulty. Based on that, we use a penalty weight updating strategy to calculate the class penalty weights ω, and use a curriculum reconstruction strategy to sample a balanced curriculum Φ + Φ∗ for training the SVM classifier in the next stage.

Experiment

Evaluation on the MIT-67, Caltech-101, MNIST, and MLC datasets

Table 1. Comparison with the state-of-the-art imbalanced learning methods on the tasks of scene classification (MIT-67), object classification (Caltech-101), handwritten digit classification (MNIST) and coral classification (MLC). We randomly set 50 sampling lists of the first three datasets respectively and report the mean performance since we can not get the list in the original paper, except for the MLC.

Evaluation on the SD-198 and SD-260 datasets

Table 2. Comparison to the state-of-the-art imbalanced learning methods on both the SD-198 and SD-260 datasets under different evaluation metrics. Each entry in this table is composed of the mean and variance of the corresponding performance derived by cross-validation.

Comparison of clinical skin disease diagnosis

Table 3. Comparison results of clinical skin disease diagnosis on the SD-198 dataset. SIFT and CN (color name) are extracted by using the code of [94]. ”-ft” means fine-tuning the VggNet on SD-198. TS-L is Texture Symmetry of Lesion; CNL is Color Name of Lesion; AC-L is Adaptive Compactness of Lesion; ‘General-D” is the recognition accuracy of the general doctor who does not focus on one specific kind of disease; “Junior-D” is the recognition accuracy of junior dermatologist; C-Int is the intergeneration of three kinds of representations TS-L, CN-L, and AC-L.

Accuracy gains on the SD-198 dataset

Figure 3. Accuracy gains of SPBL over the comparative methods on the SD-198 dataset. (a) The class IDs are ranked by the instance number of categories from large to small, which are drawn by the red line. (b) The class IDs are ranked by the recognition difficulty (calculated by Eq. 6) of categories from easy to difficult, which are drawn by the red line. For both sub-figures, Y-axis (left) indicates the accuracy gains of SPBL against the other four methods. Y-axis of (a) (top right) refers to instance number of each class. Y-axis of (b) (bottom right) is the recognition difficulty of each class.

t-SNE visualizations

Figure 4. Visualizations of 2D t-SNE [98] feature embedding on the SD-198 (a-b) and SD-260 (c-d) datasets. (a) and (c) are the feature embedding using the features extracted from the raw ResNet50, i.e., trained using all samples without the consideration of class imbalance and the SPL paradigm, on the SD-198 and SD-260 datasets, respectively. (b) and (d) are the feature embedding by using the features derived from the model of SPBL trained on SD-198 and SD-260, respectively. Note the models in all figures are trained with the same number of epochs.