State of the Art in Neural Networks and Their Applications: Volume 1

State of the Art in Neural Networks and Their Applications presents the latest advances in artificial neural networks and their applications across a wide range of clinical diagnoses. Advances in the role of machine learning, artificial intelligence, deep learning, cognitive image processing and suitable data analytics useful for clinical diagnosis and research applications are covered, including relevant case studies. The application of Neural Network, Artificial Intelligence, and Machine Learning methods in biomedical image analysis have resulted in the development of computer-aided diagnostic (CAD) systems that aim towards the automatic early detection of several severe diseases. State of the Art in Neural Networks and Their Applications is presented in two volumes. Volume 1 covers the state-of-the-art deep learning approaches for the detection of renal, retinal, breast, skin, and dental abnormalities and more.

• Includes applications of neural networks, AI, machine learning, and deep learning techniques to a variety of imaging technologies
• Provides in-depth technical coverage of computer-aided diagnosis (CAD), with coverage of computer-aided classification, Unified Deep Learning Frameworks, mammography, fundus imaging, optical coherence tomography, cryo-electron tomography, 3D MRI, CT, and more
• Covers deep learning for several medical conditions including renal, retinal, breast, skin, and dental abnormalities, Medical Image Analysis, as well as detection, segmentation, and classification via AI

• Language: English
• Publisher: Academic Press
• Release date: Jul 21, 2021
• ISBN: 9780128218495

Book preview

Chapter 1

Computer-aided detection of abnormality in mammography using deep object detectors

Pengcheng Xi¹,², Ghazal Rouhafzay¹,³, Haitao Guan⁴, Chang Shu¹, Louis Borgeat¹ and Rafik Goubran²,    ¹1National Research Council Canada, Ottawa, ON, Canada,    ²2Carleton University, Ottawa, ON, Canada,    ³3University of Ottawa, Ottawa, ON, Canada,    ⁴4Nantong No.3 People's Hospital, Nantong, China

Abstract

Computer-aided detection of medical abnormalities is of great importance for both doctors and patients. In this chapter, we use mammogram screening as an example and introduce two state-of-the-art deep neural networks for the detection of mass tissues. More specifically, we compare two-stage and one-stage object detectors using deep convolutional neural networks. With the limited number of training data, we use transfer learning to fine-tune the general object detectors on a publicly available mammogram dataset. Experimental results indicate that the deep learning-based approaches have great potential in mammogram screening.

Keywords

Mammography; deep learning; convolutional neural networks; transfer learning

1.1 Introduction

According to a study by the World Health Organization, breast cancer is the most common cancer in women [1]. Around the world, the incidence of breast cancer is increasing due to life expectancy increase and urbanization. Despite preventative measures, early detection remains the cornerstone for improving patients’ outcome and survival from breast cancers.

Among existing imaging techniques, mammography is common for breast screening. It uses a low-dose X-ray system to examine the breasts and remains the most reliable method for screening [2]. For patients who have demonstrated abnormal clinical findings, diagnostic mammography needs to be done as a follow-up examination [2]. In mammography, normally two views for each breast are used: the craniocaudal (CC) and the mediolateral oblique (MLO) views.

Screening mammograms have challenges including low contrast in the images. Double reading has been used to reduce the rate of false-positive and false-negative detections, but the cost is high [3]. Accordingly, computer-aided detection (CADe) is playing an increasingly important role [4]. CADe is a pattern recognition process that assists radiologists in detecting abnormalities, including architectural distortions, mass, or calcification in mammograms [5].

There is an abundance of CADe methods developed for medical abnormalities [2,4,6,7]. In order to detect subtle and crucial image details, traditional methods rely on a manual process of carefully designing image features, whereas recent deep learning methods conduct automatic feature learning. Due to a large amount of training data, the deep learning approaches have achieved great success in image classification and object detection [8–11]. The deep learning approaches are robust to dataset noise, making them suitable for abnormality detection.

In this chapter, we present several abnormality detection approaches for CADe in mammography using deep convolutional neural networks (CNNs). We introduce two deep learning models for object detection, namely faster R-CNN and You Only Look Once (YOLO), and compare their performance on a publicly available mammogram dataset. In particular, we fine-tune pretrained deep CNNs for the detection of mammogram abnormality using transfer learning [12,13]. Experimental results indicate that the fine-tuned deep CNN models have great potential in the CADe of mammogram abnormality.

This chapter is organized as follows. In Section 1.2, we review CADe approaches using state-of-the-art deep learning approaches. In Section 1.3, we introduce deep learning approaches used for the detection of mammogram abnormalities. In Section 1.4, we present experimental results on training and testing the deep learning models on a mammogram dataset. In Section 1.5, we discuss on the results. Section 1.6 concludes the chapter with future work plans.

1.2 Literature review

Deep learning-based computational architectures have witnessed a significant advancement in recent decades, making them a promising solution for many applications, especially those with high level of data complexity. Medical images contain intricate structures and details indistinguishable to human eyes, thus require a powerful tool to analyze their features. Many researchers have taken advantage of deep learning-based architectures, which are capable of learning and capturing such complex structures for medical image analysis.

Deep learning has been applied to the segmentation of important structures from medical images [14,15]. In Ref. [14], the state-of-the-art performance was achieved on retinal vessel segmentation using a multiscale multilevel CNN. A side-output layer was used to learn hierarchical representation of features. In Ref. [15], a new deep learning architecture was proposed for segmenting optic disk and optic cup from fundus images. The segmentation was achieved through a one-stage join multilabel system, comprising U-shape network, side-output layer, multiscale input layer, and multilabel loss function.

Deep learning-based architectures have shown good performance on the detection of small abnormalities. In a recent work, Zhang et al. [16] detected lesions in computational tomography (CT) kidney images using a faster R-CNN [17]. They took advantage of two morphological convolution layers to ease the procedure of lesion detection by eliminating noise. Moreover, they introduced a modified feature pyramid network in the first module (RPN) of faster R-CNN and combined four cascade R-CNNs in the second module to enable the detection of small lesions.

The high rate of breast cancer and the importance of early diagnosis have encouraged many researchers to devote an enormous effort to the analysis of mammogram images. Deep learning has been applied to breast lesion detection [18,19]. It has also been used for the detection and classification of breast abnormalities in mammograms [20]. In Ref. [20], the authors built their system based on YOLO [21], which contains four main stages: mammogram preprocessing, feature extraction using multiple convolutional layers, mass detection, and classification using a fully connected neural network. During training, the regions of interests (ROIs) and types of masses are provided with the mammogram images to train a YOLO model. The model is then used to detect masses in a testing image and classify them as benign or malignant.

There is a necessity of having a common public database for the evaluation and comparison of different approaches. This need has motivated the authors in Ref. [22] to produce and publish the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM). Since the introduction of this dataset, a series of researches have been conducted.

Based on CBIS-DDSM, our previous work [23] trained deep learning classifiers on abnormality patches and integrated them with class activation mappings (CAM) [24]. The fine-tuned CAM model is then applied to a test image for creating a color-coded abnormality map, highlighting possible abnormalities in the image. Recently, Tsochatzidis et al. [25] adopted seven different CNN architectures to classify mass data from the CBIS-DDSM dataset into benign or malignant. They compared the results through fine-tuning the parameters of a pretrained network. Chougard et al. [26] performed a multilabel classification using a pretrained VGG16 model [9] and proposed a solution to fine-tune the network in order to achieve optimum performance. Li et al. [27] developed a deep learning-based approach for breast cancer segmentation and diagnosis, where they introduced a pixel-wise loss function. The authors tested their method on four different datasets including CBIS-DDSM, where they obtained a dice coefficient of 50.91%. Agarwal et al. [28] detected mass in the CBIS-DDSM database where they trained a CNN on mass positive patches versus mass negative patches extracted from full mammogram images. The trained network was then used for mass detection from full mammograms by passing several extracted patches from the full test image through the trained network.

In summary, traditional CADe approaches rely heavily on manual features, which are known to be time-consuming and error-prone. In comparison, deep learning CADe replaces this process with automatic feature learning from data. It provides a complete joined solution through the integration of feature extraction and object detection.

1.3 Methodology

Abnormality detection in medical images is essentially an object detection problem; therefore we can leverage the approaches that have been developed for general object detections. The state-of-the-art deep learning object detectors can be categorized into two types: two-stage or single-stage networks. A two-stage network first computes region proposals in the images and then classifies the proposed image regions for object detection. Examples of this type include regions with CNN features (R-CNN) and its variants (fast and faster R-CNNs). In comparison, a single-stage network makes predictions on the whole image, which are then decoded to generate bounding boxes for detected objects. Examples of this type include YOLO and its variants (YOLO V2 and SSD).

There are challenges in applying general object detection techniques to the medical domain. Deep learning-based detectors need a large amount of training data, whereas medical image labeling requires lots of expertise and therefore it is costly and time-consuming. Different from general object detectors where mostly natural images are used for training, the challenge with mammography is that the abnormalities are located in tiny local regions within a high-resolution medical image.

In order to address these challenges, this chapter proposes two types of approaches based on deep neural networks (DNNs). The first type is based on the combination of region proposals with object detection networks. A representative model is faster R-CNN, in which the region proposal and object detection share convolutional layers for feature computation. It achieves good speed and high precision on object detection. The second type combines the region proposal network (RPN) with object detection into one single step. It runs a single convolutional network on the image and thresholds the resulting detections by the model’s confidence. YOLO and its improved models belong to this type.

1.3.1 Architectures of deep convolutional neural networks and deep object detectors

The success of deep learning in computer vision owes to the availability of large datasets with annotations. One notable example is ImageNet Large Scale Visual Recognition Competition [29], which contains over 15 million annotated images from a total of over 22,000 categories. Over the years of this competition, deep CNNs kept refreshing the top performance of image classification and object detection. An incomplete list of these models includes AlexNet [8], VGGNet [9], GoogLeNet [10], and ResNet [30]. While most recent models have achieved better results, their depth and complexity are also increasing. In order to deploy these models to mobile devices, a recent model called MobileNet v2 [31] was proposed. In this chapter, we choose a subset of these models for training and testing abnormality detections.

Among deep learning object detectors, the two-stage deep learning (DL) object detectors include R-CNN [32], fast R-CNN [33], and faster R-CNN [17]. They share in common that rectangular region proposals are integrated with deep CNNs for object detection. These approaches first search for region proposals from input images that may contain objects and then compute CNN features on these regions. Then, they classify the objects and adjust the boundaries of the region proposals.

1. R-CNN—This approach combines rectangle region proposals and CNNs. The first step is to compute a set of regions that may contain objects, and the second step learns classifiers to classify the regions. The region proposals are generated using edge boxes. The proposals are then cropped and resized to train CNN classifiers. Deep CNNs are used for feature extraction to be plugged into a support vector machine to refine the region proposal bounding boxes.

2. Fast R-CNN—This approach is similar to R-CNN in that it uses algorithms such as edge boxes to generate region proposals. Instead of cropping the proposal regions and resizing them, this approach works on the entire image by pooling CNN features corresponding to each region proposal. This is a more efficient implementation, because it shares computation with overlapping regions.

3. Faster R-CNN—This approach makes further improvement by discarding external algorithms such as the edge boxes. It merges the step of region proposals into the entire network and makes computations more efficient.

The three methods share the same component of R-CNN for object detection. Faster R-CNN makes great improvements over the other two in that it does not rely on external approaches for computing region proposals, which was seen as a bottleneck. After faster R-CNN, several object detectors have been invented for improving efficiency, such as single-shot detector (SSD) [34] and region-based fully convolutional neural networks [35]. However, a recent study demonstrates that faster R-CNN is still competitive on accuracy among the three [36]. It suggests that using fewer proposals, faster R-CNN can speed up significantly without a big loss in accuracy. Therefore in this chapter, we choose faster R-CNN as the detector and apply transfer learning [13] to adapt it to the medical dataset.

Besides the two-stage DL object detectors, the one-stage DL detectors include YOLO [21], YOLO V2 [37], YOLO 9000 [37], and SSD [34]. Instead of using cascaded steps for region proposal prediction and object detection, these models use one single network for the tasks.

1. YOLO—It frames object detection as a regression problem to spatially separated bounding boxes and associated class probabilities. It uses a single-neural network, which predicts bounding boxes and class probabilities directly from full images in one evaluation. It can be optimized end-to-end on detection performance.

2. YOLO v2—It makes various improvements to the YOLO detection method to make it better and faster. The improvements include batch normalization, pretraining a high-resolution classifier, predicting offsets rather than coordinates, computing dimension clusters, multiscale training, and so on.

3. YOLO 9000–It further improves the YOLO v2 model to be stronger through a joint training algorithm to train a model on more than 9000 classes from ImageNet as well as detection data from COCO.

4. SSD—It discretizes the output space of bounding boxes into a set of default boxes over different aspect ratios and scales per feature map location. At prediction time, the network generates scores for the presence of each object category in each default box and produces adjustments to the box to better match the object shape. In addition, the network combines predictions from multiple feature maps with different resolutions to naturally handle objects of different sizes.

According to Ref. [37], YOLO v2 achieved the highest mean average precision (mAP) of 78.6 on COCO dataset, in comparison to 76.8 achieved by SSD and 76.4 by faster R-CNN. Because of the performance difference, we select YOLO v2 as the abnormality detector in the chapter.

For a limited target dataset, transfer learning has proved to be effective in reusing deep learning models pretrained on large-scale datasets [13]. The idea of transfer learning for classification also applies to object detection. All DL object detectors rely on convolutional layers for feature computation, and those layers can be directly transferred from existing deep CNNs that have been trained on large-scale datasets such as the ImageNet. This can effectively reduce the amount of training data that are needed to train a DL-based medical abnormality detector.

1.3.2 Abnormality detection with faster R-convolutional neural networks

Faster R-CNN comprises two modules, the RPN and the fast R-CNN detector that uses the proposed regions from the RPN. Fig. 1.1 illustrates the layout of a medical abnormality detector using the faster R-CNN framework. Given an image input, convolutional layers are used to compute feature maps. Based on the computed feature maps, the RPN is trained to propose ROIs. ROI pooling is then used to combine the feature maps and the output from RPN for predicting the classes of objects and adjusting the bounding boxes that contain these objects.

Figure 1.1 Faster R-CNN for detecting medical abnormality in mammogram images. R-CNN, convolutional neural networks.

The RPN tells the fast R-CNN where to look, through applying an attention mechanism to the input image. It includes a pretrained deep fully convolutional network, which outputs rectangular region proposals for possible objects as well as their associated object-ness scores. In particular, the RPN uses the last shared convolutional layer of a pretrained CNN as the feature map. It then applies an additional convolutional layer in order to map each window in the feature map into lower dimensional features, when the convolution window slides over the feature map. These features are then fed into two parallel fully connected layers, one computing the box regression and the other classifying the box. To make the learning task easier, faster R-CNN introduces some reference rectangular regions (named anchors) centered at each sliding window and learns the offset between these reference boxes and the proposal rectangles during training. The anchors have predetermined scales (128, 256, 512) and aspect ratios (1:1, 1:2, 2:1).

Training a faster R-CNN comprises alternating steps between training an RPN and a fast R-CNN, because the latter two share feature computations. The process starts from training an RPN and uses the proposed regions to train a fast R-CNN. The weights from the fast R-CNN are then shared with retraining the RPN. The updated RPN is used again to further train the fast R-CNN. As a result, both the RPN and the fast R-CNN use the shared convolutional layers to build a unified network as a faster R-CNN [17].

1.3.3 Abnormality detection with YOLO

YOLO [21] is another deep learning-based object detector that we choose for medical abnormality detection. YOLO relies on a single-stage network for object detection, allowing it to work faster than the two-stage object detectors at the expense of being less accurate. YOLO is developed to imitate the working principle of human glance, which is able to extract a huge amount of information from images instantly, such as object classes and their locations. Using a single DNN, it directly determines bounding box coordinates as well as their class probability from image pixels. One of the most valued advantages of YOLO is that it generalizes efficiently to new domains and thus has potential for biomedical image analysis; however, the first version of YOLO has strong spatial constraints on bounding boxes, limiting the number of detectable nearby abnormalities. Moreover, YOLO fails to generalize to unusual aspect ratios and configurations. Therefore the trained detector will not work efficiently for a new dataset, where the sizes of abnormalities are very different.

Since 2015, two updated versions of YOLO have been introduced, including YOLO v2 [37] and v3 [38]. In YOLO v2 the authors try to enhance recall and reduce the localization error. Inspired by the anchors used in faster R-CNN, in YOLO v2, the fully connected layers are removed and a set of predetermined anchors are given to the network, such that the network learns to predict offsets for bounding boxes instead of their coordinates. Unlike YOLO where each grid can only detect one object, introducing the anchors allows the detection of more than one object within a grid. YOLO v2 can also be run at a variety of image sizes.

YOLO v3 detects objects at three different scales down-sampled from the original image by factors of 32, 16, and 8. It takes advantage of the custom DarkNet-53 with 53 layers trained on ImageNet. Fifty-three extra layers are added for object detection in order to build up the overall 106-layer architecture of YOLO v3. In YOLO and YOLO v2 the loss function computes squared errors for penalization terms, which are substituted by cross-entropy in YOLO v3. Furthermore, YOLO v3 removes the soft-max layer, enabling the model to perform multilabel classification for detected objects.

Fig. 1.2 summarizes the overall framework of YOLO. The input image is first divided into a SxS grid. For each grid, it predicts multiple bounding boxes and confidence of these boxes. In parallel the model also predicts the probability that the grid corresponds to an abnormal tissue (mass). The outputs are then combined for determining the four parameters [bx, by, bh, bw], specifying bounding box locations for detected abnormal tissue.

Figure 1.2 YOLO model for detecting medical abnormality in mammogram images. YOLO, You Only Look Once.

1.4 Experimental results

1.4.1 Data preparation

This research is conducted over CBIS-DDSM [22], including the left and right human breasts in MLO and CC views. It contains a total of 3013 mammograms with two types of abnormalities, namely calcification and mass. The dataset is split into 1227 calcification cases for training, 284 calcification cases for testing, 1231 mass cases for training, and 361 mass cases for testing. For each mammogram image, the dataset also provides a set of abnormal regions (ROI) in the form of a binary image of the same size as the mammogram. In the binary image, abnormalities are marked as white regions on a black background (see Fig. 1.3B). In this chapter, we only select the mass data for training and testing abnormality