Introduction
Breast cancer has intention to spread in the body. Cancer cells spread locally by spreading to the nearest normal tissue. And spread regionally by moving to nearby lymph nodes, tissues or organs. At advanced stage, cancer can also spread to distant parts of the body, which is called as metastatic cancer.
Under microscopic observation and diagnostic testing, metastatic cancer cells have similar features of the primary cancer (from where it is started) and generally not like the cells in the place where it is found. This facilitates the physicians to identify whether it is metastatic or local tissue cancer. Although metastatic breast cancer (MBC) is unlikely to be cured, meaningful improvements in survival have been seen, coincident with the introduction of newer systemic therapies.1, 2, 3 Median overall survival now is slightly over three years, with a range from a few months to many years.4
When cancer cells break away from the primary cancer, they can travel to other areas of the body either through the bloodstream or through the lymph system. The lymph nodes, under arm, inside the breast and the nearest collarbone are among the first places where breast cancer spreads. Tumor cells migrating from a primary tumor, metastasize to one or a few lymph nodes before involving others. The entire breast drains, the tumor cells to the same few lymph nodes, regardless of injection site.5 These few lymph nodes are called "sentinel" lymph nodes. The status of sentinel lymph nodes would accurately predict the status of the remaining lymph nodes.
Lymph node assessment is considered to be one of the most independent prognostic factors in the breast cancer. It is also an important component of the TNM breast cancer staging system. In TNM staging system, patients with a sentinel lymph node positive will receive a higher pathologic TNM stage for sentinel lymph node metastasis than the patients with negative. This higher pathologic TNM staging frequently results in more aggressive clinical management including axillary lymph node dissection.6, 7, 8
Assessing lymph node metastasis through histologic examination is the most accurate method. Each WSI image is gigapixels in size and analyzing the entire image manually is extremely difficult which consumes lot of pathologist’s time. Due to the time-consuming and tedious nature of the diagnosis of breast cancer, there is a growing need for it to be automated.
Convolutional Neural Networks (CNNs) are shown excellent performance and gaining extensive attention in digital image processing tasks such as image classification, segmentation, object detection tasks etc.9 CNNs are the most well-known type of deep learning architecture which perform well in the medical image analysis field. These CNN models which can automatically learn complex medical images characteristics and provide quantitative measures which facilitates increased accuracy in disease diagnosis with higher efficiency.
This paper envisions a new and faster sentinel lymph metastases classification model which will help the pathology experts to perform fast and accurate diagnosis. This paper uses a CNN model called Densenet-161, a novel version of DenseNet model with 161 layers to classify breast cancer metastases in the sentinel lymph node WSI.10 The experimental PCam dataset contains 327,680 patches extracted from Cameylon16 Challenge dataset at a size of 96x96 pixels @10x magnification.11 The experimental dataset is derived from 400 H&E stained breast cancer SLN WSIs of size 97792 x 220672 pixels.
Materials and Methods
Technical Information
CNNs contain stack of various layers like input layer, a hidden layer and an output layer. The hidden layer of CNNs generally contains one or more convolutional layers, pooling layers, and fully connected layers.
Recently large-scale image analysis tasks especially complex medical image analysis tasks have shown that CNNs which contain shorter connections between layers close to the input and output, can be significantly deeper, more accurate and more efficient to train. The Dense Convolutional Network (DenseNet) leverages this observation and connects each layer with each of its previous layers in a feed-forward fashion. Traditional L-layers CNNs have L-connections, one connection between each layer and its subsequent layer. The number of connections in DenseNet is L*(L+1)/2. DenseNet keeps the feature-maps of all preceding layers as inputs to the current layer. All layers are connected so that the flow of information between layers in the network is maximum. There are multiple versions of Densenet in an L-layered network. This paper proposed Densenet-161 which consists 161 layers.
Data
Lymph nodes are oval-shaped organs found in numerous parts of the body, including the armpits, neck and groin. Sentinel lymph nodes spread cancer cells to other parts of the body as illustrated in Figure.1 The experimental dataset contains
CNN Training & Pre and Post processing Methods
The pre-processing consists of two steps. The first step consists of converting the three-channel RGB (colour) images into single-channel grayscale images. The second step is the application of truncation, which is a binary thresholding technique. Binary thresholding is done to highlight the image features i.e to make darker components darker and lighter components lighter. These single-channel images are stacked up three times to mimic three-channel RGB images.
Transfer learning technique is a machine learning technique where a model trained on one task is re-purposed on a second related task. This paper leverages the modern deep learning library Fastai for the implementation of deep learning model.12
Training images in the experimental dataset has been shuffled and get_transforms function of fastai has been applied to augment the dataset. The concerned get_transforms function applies data augmentation techniques like image resizing, random cropping etc. Data is normalized with mean and deviation values range in [0.485, 0.456, 0.406] and [0.229, 0.224, 0.225] respectively in line with ImageNet model, a precursor to DenseNet model.
The pre-trained model is loaded using a fastai library function cnn_learner, which takes data, the model name, metric type and callback function as input parameters. Another fastai library function lr. find() which performs a mock training on data and plots the learning rate vs loss relationship has been used to reduce the amount of guestimates to pick initial learning rate for the learner. The experimental data set uses a learning rate of 1e-0.2. The well-known method fit_one_cycle method has been used to fit the model with the chosen learning rate and the number of epochs as input parameters.
Same pre-processing techniques such as three-channel conversion and binary thresholding are applied to test images. A prediction function is applied to compare the annotated value with the predicted value.
Results
Generally, the CNN models have been evaluated by using the metrics, dice coefficient, Jaccard coefficient, accuracy and F1-score.13, 14 Vikas Thada et al has done detailed comparison of the metrics for the researcher’s reference.15
Accuracy is one of the most common measure to evaluate models which measures all the correctly identified cases. In few cases, model evaluated by accuracy may fail to identify small, positive, and critical imaging regions. In imbalanced datasets, a model may have good accuracy, but may fail to identify such critical regions. In such scenario, to properly access the model, dice, Jaccard coefficients and F1-score are used.
CNN model initially learns the features from the training data. Subsequently after fitting the training data well it tries to generalize and make accurate predictions for the incoming new data which it has not seen earlier. Overfitting refers to a model that models the training data too well. It implies that model learns the features and noise in the training data to the extent that it negatively impacts the performance of the model on new data. Overfitting should be avoided. Data scientists do cross-validation to find out if their model is overfitting, wherein they split their data into two parts - the training set and the validation set. The training set is used to train the model whereas the validation set is used to evaluate the model's prediction performance while tuning model hyperparameters. The testing set is data applied on the final model to provide an unbiased evaluation of a final model fit on the training dataset.
Training accuracy measures the progress of model in terms of its training. Validation accuracy measures the quality of model measured on its ability of new predictions based on data it hasn't seen before. With an accuracy of ~82% on the validation set means that model can predict with ~82% accuracy on new data. Testing accuracy provides the actual (testing) performance numbers of the final model.
The proposed system has generated a training accuracy of 0.9477 and validation accuracy of 0.944. The proposed system has correctly classified 28127 images out of 32768 images belonging to the testing dataset, with an accuracy of 85.84%, as shown inTable 1
Precision is measure of the correctly identified positive cases from all the predicted positive cases. Recall measures correctly identified positive cases from all the actual positive cases. F1-score is a harmonic mean of Precision and Recall and provides a better measure than Accuracy metric with incorrectly classified cases. In most and highly complex real-life classification problems like medical imaging analysis, imbalanced class distribution exists and thus F1-score is a better metric to evaluate the model. The proposed system has achieved a F1-score of 0.8406 as shown in Table 2
Discussion
Initially, approaches to pathological image analysis in digital pathology have primarily focused on low level image analysis tasks like color normalization, nuclear segmentation, and feature extraction. Subsequently, classification models construction using machine learning methods like regression, support vector machines, and random forests are evolved over a period. Later on Deep Learning models, CNNs are gaining attention in complex medical image analysis tasks like image classification, image segmentation, object detection etc, which are part of disease diagnostic pipelines.
This paper discusses few of the well-researched, established and implemented CNN models on digital pathology imaging techniques.
Anant Madabhushi et al explains out performing various computational tumoral markers identification tasks like Carcinoma Localization, Nuclei Segmentation, Epithelium Segmentation, Tubule Segmentation, Lymphocyte detection, Mitosis Detection and Lymphoma classification on WSI pathology images by implementing various Deep Learning Techniques.16 Authors explained-out epithelium segmentation task as an use case. Authors have used 34 digital pathology 1000x1000 pixels training images and 8 validation images and performed epithelium segmentation from WSI images. Multiple patches of size 32x32 are extracted from each WSI image and each patch containing the edges of epithelium regions are taken so that the network can learn crisp boundaries. The popular CNN algorithm, AlexNet is applied to classify the images on the generated patches. White regions are removed by applying user-defined thresholding and positive regions which aren’t clinically relevant, of size less than 300 pixels are removed to generate the output. An average F1 score of 0.84 has generated by this method.
Muhammed Tao has introduced DenseNet-161 and ResNet-50 pre-trained CNN models and classified into multiple classes of WSI images, based on textural patterns of images by performing transfer learning on those images.17 The author has used 23,916 images of size 1000x1000 pixels, which are extracted from 24 WSI images of various body parts with different texture patterns. Author has tried to prove that automated detection and classification of diseases using computational scanned microscope images with much higher accuracy and faster time. By performing the transfer learning of the pre-trained models DenseNet-161 and ResNet-50 on digitized histopathology images has achieved a classification accuracy of 97.89% using grayscale images and the ResNet-50 model obtained the accuracy of 98.87% for color images. Author has shown that the pre-trained models used in this experiment outperform state-of-the-art methods in all image classification performance metrics with 24 image categories.
Wang et al have evaluated classification performance of four different existing deep learning networks of the time for the classification of SLN with MBC. Proposed a GoogleNet based deep learning network for this classification task.18 The researchers have implemented four well-known deep learning networks of the time namely, GoogleNet, AlexNet, VGG16 and a face oriented deep network to classify SLN with MBC. Subsequently, they used GoogleNet in their deep learning framework for their patch-based classification task. And also established that combining deep learning networks with pathologist’s prediction have reduced the pathologist error rate from over 3 percent to less than 1 percent.
Based on observation of digital pathology images which are inherently symmetric under rotation and reflection B.S.Veeling et al proposed new deep learning model for pathology images segmentation.19 B.S.Veeling model leverages inherent symmetries of pathology images in a principled manner and shown that model has improved stability on predictions. Also demonstrated that exploiting rotation equivariance improves the performance of tumor detection on lymph node metastases dataset significantly. The proposed model, a patch-classification model is derived from the densely connected convolutional network (DenseNet). The DenseNet is primarily consisting dense blocks with layers which uses stack of all previous layers as input, alternating with 1 x 1 convolutional layer as transition blocks and 2 x 2 strided average pooling. Presented a novel large-scale pathology images derived dataset for precise machine learning model evaluation. Patients with a sentinel lymph node positive will receive a higher pathologic TNM metastasis breast cancer stage than patients negative for sentinel lymph node metastasis which frequently results in more aggressive clinical management and treatment including axillary lymph node dissection.
This paper discussed a CNN based image classification model, to classify breast lymph node metastasis from WSI images, called DenseNet-161. The DenseNets have substantial advantages over traditional CNNs, that they reduce the vanishing-gradient problem, feature reusage, strengthen feature propagation, significant reduction in number of parameters and less computation time. DenseNet with its novel architecture of connecting each of its layer to each of their previous layers in a feed-forward fashion has substantial advantage over traditional CNNs. Traditional data augmentation techniques like image resizing, random cropping etc. are used to artificially increase the dataset size to obtain a better fit on the model. This paper also discussed about pre-processing techniques like binary thresholding to avoid the information loss. This paper also discussed about the modern library fastai which is used to perform transfer learning on the experimental dataset.
Each breast WSI image is in gigapixels size and analyzing gigapixels image manually is extremely difficult and time consuming to pathologists. As diagnosing the breast cancer is time-consuming and tedious, there is a growing need to automate this process. This paper envisions an image classification model to classify breast lymph node metastasis on WSI image which facilitate pathology experts to perform fast and accurate diagnosis task.
This model involves extraction of complex information from the medical images dataset, which requires the removal of noise. Even after applying thresholding pre-processing method the noise persists, which requires additional pre-processing before training the model. And by increasing the dataset size through data-augmentation will also improve the accuracy considerably.
Acknowledgement
Dr.Rohit Tapadia, MBBS, MD, Director, Tapadia Diagnostic Centre, Hyderabad for Biomedical, Clinical Advisory and Evaluation.
Article DOI
