Radiomics and Its Clinical Application: Artificial Intelligence and Medical Big Data

By Jie Tian, Di Dong, Zhenyu Liu and Jingwei Wei

The rapid development of artificial intelligence technology in medical data analysis has led to the concept of radiomics. This book introduces the essential and latest technologies in radiomics, such as imaging segmentation, quantitative imaging feature extraction, and machine learning methods for model construction and performance evaluation, providing invaluable guidance for the researcher entering the field.

It fully describes three key aspects of radiomic clinical practice: precision diagnosis, the therapeutic effect, and prognostic evaluation, which make radiomics a powerful tool in the clinical setting.

This book is a very useful resource for scientists and computer engineers in machine learning and medical image analysis, scientists focusing on antineoplastic drugs, and radiologists, pathologists, oncologists, as well as surgeons wanting to understand radiomics and its potential in clinical practice.

• An introduction to the concepts of radiomics
• In-depth presentation of the core technologies and methods
• Summary of current radiomics research, perspective on the future of radiomics and the challenges ahead
• An introduction to several platforms that are planned to be built: cooperation, data sharing, software, and application platforms

• Language: English
• Publisher: Academic Press
• Release date: Jun 3, 2021
• ISBN: 9780128181027

Jie Tian

Dr. Jie Tian received his PhD (with honors) in Artificial Intelligence from the Chinese Academy of Sciences in 1993. Since 1997, he has been a Professor at the Chinese Academy of Sciences. Dr. Tian has been elected as a Fellow of ISMRM, AIMBE, IAMBE, IEEE, OSA, SPIE, and IAPR. He serves as an editorial board member of Molecular Imaging and Biology, European Radiology, IEEE Transactions on Medical Imaging, IEEE Transactions on Biomedical Engineering, IEEE Journal of Biomedical and Health Informatics, and Photoacoustics. He is the author of over 400 peer-reviewed journal articles, including publication in Nature Biomedical Engineering, Science Advances, Journal of Clinical Oncology, Nature Communications, Radiology, IEEE Transactions on Medical Imaging, and many other journals, and these articles have received over 25,000 Google Scholar citations (H-index 79). Dr. Tian is recognized as a pioneer and leader in the field of molecular imaging in China. In the last two decades, he has developed a series of new optical imaging models and reconstruction algorithms for in vivo optical tomographic imaging, including bioluminescence tomography, fluorescence molecular tomography, and Cerenkov luminescence tomography. He has developed new artificial intelligence strategies for medical imaging big data analysis in the field of radiomics and played a major role in establishing a standardized radiomics database with more than 100,000 cancer patients data collected from over 50 hospitals all over China. He has received numerous awards, including 5 national top awards for his outstanding work in medical imaging and biometrics recognition.

Book preview

Radiomics and Its Clinical Application

Artificial Intelligence and Medical Big Data

Jie Tian

CAS Key Laboratory of Molecular Imaging, Chinese Academy of Sciences; Beijing Advanced Innovation Center for Big Data-Based Precision Medicine, School of Engineering Medicine, Beihang University, Beijing, China

Di Dong

CAS Key Laboratory of Molecular Imaging, Chinese Academy of Sciences; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

Zhenyu Liu

CAS Key Laboratory of Molecular Imaging, Chinese Academy of Sciences; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

Jingwei Wei

CAS Key Laboratory of Molecular Imaging, Chinese Academy of Sciences; School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

Table of Contents

Cover image

Title page

The Elsevier and Miccai Society Book Series

Copyright

Biographies

Preface

Chapter 1. Introduction

1.1. Background of medical image analysis in cancer

1.2. Multidimensional complexity of biomedical research

1.3. Concept of radiomics

1.4. Value of radiomics

1.5. Workflow of radiomics

1.6. Prospect of clinical application of radiomics

Chapter 2. Key technologies and software platforms for radiomics

2.1. Tumor detection

2.2. Tumor segmentation

2.3. Feature extraction

2.4. Feature selection and dimension reduction

2.5. Model building

2.6. Radiomics quality assessment system

2.7. Radiomics software platform

Chapter 3. Precision diagnosis based on radiomics

3.1. Application of radiomics in cancer screening

3.2. Application of radiomics in cancer staging

3.3. Application of radiomics in histopathological diagnosis of cancer

3.4. Application of radiomics in prediction of cancer gene mutation and molecular subtype

3.5. Application of radiomics in other diseases

Chapter 4. Treatment evaluation and prognosis prediction using radiomics in clinical practice

4.1. Radiomics and its application in treatment evaluation

4.2. Radiomics-based prognosis analysis

Chapter 5. Summary and prospects

5.1. Summary

5.2. Prospect

5.3. Conclusion

Index

The Elsevier and Miccai Society Book Series

Advisory Board

Nicholas Ayache

James S. Duncan

Alex Frangi

Hayit Greenspan

Pierre Jannin

Anne Martel

Xavier Pennec

Terry Peters

Daniel Rueckert

Milan Sonka

Jay Tian

S. Kevin Zhou

Titles

Balocco, A., et al., Computing and Visualization for Intravascular Imaging and Computer Assisted Stenting, 9780128110188.

Dalca, A.V., et al., Imaging Genetics, 9780128139684.

Depeursinge, A., et al., Biomedical Texture Analysis, 9780128121337.

Munsell, B., et al., Connectomics, 9780128138380.

Pennec, X., et al., Riemannian Geometric Statistics in Medical Image Analysis, 9780128147252.

Trucco, E., et al., Computational Retinal Image Analysis, 9780081028162.

Wu, G., and Sabuncu, M., Machine Learning and Medical Imaging, 9780128040768.

Zhou S.K., Medical Image Recognition, Segmentation and Parsing, 9780128025819.

Zhou, S.K., et al., Deep Learning for Medical Image Analysis, 9780128104088.

Zhou, S.K., et al., Handbook of Medical Image Computing and Computer Assisted Intervention, 9780128161760.

icon

Copyright

Academic Press is an imprint of Elsevier

125 London Wall, London EC2Y 5AS, United Kingdom

525 B Street, Suite 1650, San Diego, CA 92101, United States

50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States

The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

Copyright © 2021 Elsevier Inc. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN: 978-0-12-818101-0

For information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Publisher: Mara Conner

Acquisitions Editor: Tim Pitts

Editorial Project Manager: Chiara Giglio

Production Project Manager: Prasanna Kalyanaraman

Cover Designer: Victoria Pearson

Typeset by TNQ Technologies

Biographies

Dr. Jie Tian received his PhD (with honors) in Artificial Intelligence from the Chinese Academy of Sciences in 1993. Since 1997, he has been a Professor at the Chinese Academy of Sciences. Dr. Tian has been elected as a Fellow of ISMRM, AIMBE, IAMBE, IEEE, OSA, SPIE, and IAPR. He serves as an editorial board member of Molecular Imaging and Biology, European Radiology, IEEE Transactions on Medical Imaging, IEEE Transactions on Biomedical Engineering, IEEE Journal of Biomedical and Health Informatics, and Photoacoustics. He is the author of over 400 peer-reviewed journal articles, including publication in Nature Biomedical Engineering, Science Advances, Journal of Clinical Oncology, Nature Communications, Radiology, IEEE Transactions on Medical Imaging, and many other journals, and these articles have received over 25,000 Google Scholar citations (H-index 79). Dr. Tian is recognized as a pioneer and leader in the field of molecular imaging in China. In the last two decades, he has developed a series of new optical imaging models and reconstruction algorithms for in vivo optical tomographic imaging, including bioluminescence tomography, fluorescence molecular tomography, and Cerenkov luminescence tomography. He has developed new artificial intelligence strategies for medical imaging big data analysis in the field of radiomics and played a major role in establishing a standardized radiomics database with more than 100,000 cancer patients data collected from over 50 hospitals all over China. He has received numerous awards, including 5 national top awards for his outstanding work in medical imaging and biometrics recognition.

Dr. Di Dong is currently an Associate Professor at the Institute of Automation, Chinese Academy of Sciences. He received his PhD in Pattern Recognition and Intelligent Systems from the Institute of Automation, Chinese Academy of Sciences, China, in 2013. Dr. Dong is a member of the Youth Innovation Promotion Association of the Chinese Academy of Sciences, an active member of the American Association for Cancer Research (AACR), and a corresponding member of the European Society of Radiology (ESR). Dr. Dong has carried out long-term research work in the field of tumor radiomics and medical big data analysis. In recent years, Dr. Dong has published nearly 50 peer-reviewed papers in SCI journals, e.g., in Annals of Oncology, European Respiratory Journal, Clinical Cancer Research (three publications), BMC Medicine, etc. These articles have received over 1,600 Google Scholar citations (H-index 24). He has 6 ESI highly cited papers. He has applied for more than 20 patents and 10 software copyright licences in China.

Dr. Zhenyu Liu is currently a Professor at CAS Key Laboratory of Molecular Imaging, Institute of Automation. He received his PhD in Pattern Recognition and Intelligent Systems from the Institute of Automation, Chinese Academy of Sciences, China, in 2014. Dr. Liu got the outstanding youth fund of the Natural Science Foundation of China (NSFC) and is a member of the Youth Innovation Promotion Association of the Chinese Academy of Sciences. His research focuses on medical imaging analysis, especially radiomics and its applications in oncology research. In recent years, Dr. Liu has published nearly 30 papers in peer-reviewed journals, e.g., in Clinical Cancer Research, Theranostics, EBioMedicine, Radiotherapy and Oncology, etc. These articles received over 1,300 Google Scholar citations. He also holds more than 10 patents in China.

Dr. Jingwei Wei is currently an Assistant Professor at the Institute of Automation, Chinese Academy of Sciences. Her research focuses on radiomics and its clinical application in liver diseases, liver-specific feature engineering, traditional pattern recognition classifiers, and deep learning methods implemented towards liver disease-oriented research. Her primary work includes pre-operative prediction of microvascular invasion in hepatocellular carcinoma (HCC), prognosis prediction in HCC, and non-invasive imaging biomarker development for pathological factors prediction in liver diseases. Dr. Wei has published over 20 peer-reviewed papers in SCI journals, e.g., in Liver Cancer, Liver International, Clinical and Translational Gastroenterology, etc.

Preface

With the expeditious growth of medical imaging data and the rapid advancement of artificial intelligence techniques, image-derived diagnosis and prognosis of multifold diseases has broken through the scope of conventional computer-aided diagnosis. Toward the era of intelligent analysis, a new product that combines big data of medical imaging and artificial intelligence, radiomics, has emerged.

In 2012, Professor Philippe Lambin and Professor Robert Gillies first proposed the concept of radiomics, which converts medical images such as computer tomography, magnetic resonance imaging, positron emission tomography, and ultrasound into excavable data, mines massive quantitative imaging characteristics related to diseases, and builds intelligent analysis models by artificial intelligence techniques to assist clinical diagnosis and prognosis. Radiomics originates from clinical issues and eventually returns to clinical guidance applications. It is currently one of the most important research hotspots with cutting-edge directions and has definitely shown great clinical application prospects. Up to now, many mainstream international imaging conferences, such as those of the Radiological Society of North America, the International Society for Magnetic Resonance in Medicine, and the World Molecular Imaging Congress, and clinical oncology conferences (such as those of the American Association for Cancer Research, American Society of Clinical Oncology), have set up special sessions for radiomics. There is also a trend of rapid growth in international research papers related to radiomics year by year.

We have been following the research hotspot of radiomics for many years and have participated in the international radiomics seminars hosted by Professor Robert Gillies for six consecutive years. While witnessing the rapid development of radiomics and the endless novel methods and clinical applications, we are deeply concerned about the lacunae of books dedicated to radiomics in China. In view of this, we have systematically sorted out the radiomics technique procedures and typical clinical applications and compiled this book, hoping to attract more domestic clinical and scientific researchers to jointly launch radiomics researches and provide a potential technical tool for promoting the precise diagnosis and treatment of cancers and other diseases.

The publication of this book has received much help and support. We appreciate the National Science and Technology Academic Publication Fund (2017-H-017), the National Key Research and Development Program of China (2017YFA0205200), the National Natural Science Foundation of China (81930053), and the Science Press for their long-term strong support. This book is based on radiomics research studies accumulated over years by the Key Laboratory of Molecular Imaging of the Chinese Academy of Sciences and the preliminary work of many doctoral students, master's students, postdoctoral fellows, and young teachers. We are especially grateful to three authoritative experts in the field of radiomics, Robert Gillies, Philippe Lambin, and Sandy Napel, for writing prefaces to this book and supporting the research of radiomics in China. We thank Di Dong, Zhenyu Liu, Jingwei Wei, Zhenchao Tang, Shuo Wang, Hailin Li, Siwen Wang, Lianzhen Zhong, Mengjie Fang, Lixin Gong, Runnan Cao, Caixia Sun, Kai Sun, Dongsheng Gu, and Shuaitong Zhang for participating in the writing and organization of this book. They contributed a lot to the final completion of this book.

Jie Tian

October 2020

Chapter 1: Introduction

Abstract

This chapter is focused on the background of medical imaging, multidimensional complexity of biomedical research, concept of radiomics, value of radiomics, workflow of radiomics, and the clinical applications of radiomics. Deep learning techniques will also be explained in detail in this chapter as a new radiomics technique. Finally, this chapter concludes with the prospect of clinical application of radiomics.

Keywords

Artificial intelligence; Cancer; Deep learning; Machine learning; Medical imaging; Radiomics

1.1 Background of medical image analysis in cancer

1.2 Multidimensional complexity of biomedical research

1.3 Concept of radiomics

1.4 Value of radiomics

1.5 Workflow of radiomics

1.5.1 Image acquisition and reconstruction

1.5.2 Image segmentation

1.5.3 Feature extraction and selection

1.5.4 Database and data sharing

1.5.5 Informatics analysis

1.5.6 Medical image acquisition

1.5.7 Segmentation of the tumor

1.5.8 Tumor image phenotype

1.5.9 Clinical prediction for tumor

1.5.10 New technology of artificial intelligence

1.6 Prospect of clinical application of radiomics

References

Medical imaging began in 1895 when German physicist Wilhelm Konrad Rontgen discovered X-rays. In 1978, G.N. Hounsfield published computed tomography (CT) technology, which is considered to be one of the great achievements of science and technology in the 20th century. Since then, medical imaging has developed rapidly, and various new medical imaging devices and technologies have been continuously presented. Especially medical imaging has played a vital role in cancer screening, diagnosis, and treatment for patients. On the other hand, the rise of modern artificial intelligence in recent years has led to breakthroughs in computer vision and patter recognition, and the growth of massive medical big data provides an excellent opportunity for artificial intelligence applications in medical imaging analysis. Especially, the multiple molecular events represented the intrinsic progress in the micro scale, which can be possibly uncovered by artificial intelligence analysis on medical imaging. In this context, radiomics emerges as a new research area that integrates artificial intelligence and machine learning to extract pathophysiological information of tumor images, thus enabling tumor staging classification, therapeutic evaluation, and prognosis assessment. The content of this chapter is focused on the background of medical imaging, multidimensional complexity of biomedical research, concept of radiomics, value of radiomics, workflow of radiomics, and the clinical applications of radiomics.

1.1. Background of medical image analysis in cancer

According to the 2014 World Cancer Report published by the International Agency for Research on Cancer of the World Health Organization on February 3, 2014, there were 14.1 million newly diagnosed cancer cases and 8.2 million cancer-related deaths worldwide in 2012. The incidence of young patients is increasing year by year [1]. Wanqing Chen and Jie He of the National Cancer Center of China estimated there were 4.292 million newly diagnosed cancer cases and 28.14 million cancer-related deaths in China in 2015 [2]. Cancer has become a major disease that seriously affects the quality of human life and threatens human life. Early diagnosis of cancer and accurate prognosis assessment via imaging plays an important role in providing personalized treatment plans.

CT, magnetic resonance imaging (MRI), positron emission tomography (PET), and ultrasound are widely used in clinical practice. Thus, the results of relevant examinations provide assistance and reference for cancer staging and assessment of cancer patients’ prognosis. Medical imaging is an important tool for evaluation of the tumors and the curative effects. The diagnostic value of CT for cancer is that it can observe the changes of morphology and density of cancerous tissues and the extent of tumor invasion from CT images, thus achieving staging of the tumor. But it is difficult to diagnose early tumors with inconspicuous morphology. MRI has a high advantage in multidirectional imaging and soft tissue contrast ability, which can clearly show the size and location of the tumor and the degree of invasion of surrounding tissues. MRI has good clinical value for the diagnosis and staging of cancer, and can also provide information on the spread of tumors to help develop surgical plans.

As a noninvasive method for tumor diagnosis, medical imaging has been widely used in the auxiliary diagnosis of various cancers:

Firstly, the use of image information for clinical diagnosis often relies on the subjective experience of doctors, and imaging features are connected to corresponding diagnosis. However, medical imaging contained valuable information to reveal both intra- and intertumor heterogeneity. For example, based on standard medical images (e.g., CT, MRI, and PET), clinicians can only obtain concise diagnostic information related to tumor shape, size, image contrast, and tumor metabolism. However, these information may not fully reflect the pathophysiology or diagnosis and treatment of the entire tumor, and thus it is impossible to provide an effective means to quantify the imaging finding soft tumors’ pathological staging or tumor changes after treatment [3]. Medical images are not just images, in addition to providing visual information, they also contain a large amount of potential information related to tumor pathophysiology and tissue cell microenvironment [4]. Such information has not been effectively utilized for a long period of time in clinics. The deep exploration of medical imaging data will provide more information, including tumor morphology, potential pathological mechanism, and tumor heterogeneity toward the precise diagnosis and personalized treatment of patients.

Secondly, medical imaging has evolved from single X-ray imaging to a multimodal medical imaging technique. Currently, medical imaging is routinely used for clinical evaluation of tumors, diagnosis of tumor staging, and evaluation of the therapeutic effect [5,6]. However, most of the applications of these medical images only focus on the evaluation of tumor anatomical structure (e.g., tumor size) in the standardized clinical diagnosis. This measurement limits the application of medical imaging in the study of extensive tumor heterogeneity. The advantage of medical imaging is that the appearance phenotype of the tumor can be obtained in a noninvasive way, such as macroscopic intratumoral heterogeneity. Alternatively, tumor-related information can be obtained by invasive biopsy by extracting selected cancer tissue, therefore the heterogeneity of the tumor tissue cannot fully reflect the internal pathological information of the tumor. Furthermore, repeated invasive biopsies are a heavy burden for high-risk patients. Conversely, the imaging phenotype of the tumors provided by medical imaging provides a wealth of information on tumor genotypes, tumor microenvironments, and potential therapeutic effects [4]. At the same time, the information provided by the images can complement the genetic information. Therefore, the role of the tumor image phenotype based on medical imaging in precision medicine opens new possibilities for clinical research. Moreover, the clinician can quantitatively evaluate the patient’s tumor phenotype at each follow-up. Therefore, analyzing tumor heterogeneity by quantitative imaging has a great potential for precision oncology applications.

Thirdly, in current clinical practice, physicians from the radiology department use quantitative indicators for tumor evaluation. In axial CT imaging, tumor size can be described by one- or two-dimensional methods. In molecular imaging, there is less quantitative information extracted and used for clinical application. In PET imaging, only the maximum or average intake is used to quantify the metabolism. Although these indicators are highly meaningful biomarkers, a large number of potential imaging characteristics were yet to be elucidated from quantitative tumor imaging. In addition, semantic features can also be obtained from medical imaging. The semantic features refer to the tumor characteristics obtained by visual evaluation of medical images via radiologists. By definition, semantic features are qualitative judgment of the tumor in clinical practice. The extraction of semantic features highly depends on professional medical imaging knowledge and is subject to subjective influence by different human evaluators. The benefit of establishing a unified terminology standard is that the terminology defined by experienced radiologists can establish a uniform measure of tumor characterization. Moreover, radiologists can evaluate low-quality or low-resolution medical images and give diagnostic results.

1.2. Multidimensional complexity of biomedical research

Multidimensional complexity is an important problem in biomedical research, which is mainly divided into spatial complexity and time complexity.

In terms of spatial complexity, human cells contain about 20,000 protein-coding genes, about 360,000 mRNAs, and about 1,000,000 protein molecules [x]. Their biological functions and activities are difficult to measure and estimate. The human body contains about 30 trillion cells, 79 key organs, 13 major organ systems, and their phenotype and function are still difficult to estimate. The composition of the human body has a strong spatial complexity from micro to macro levels. The existing data processing methods are difficult to effectively analyze such a large amount of biomedical data.

In terms of time complexity, the interaction of each person’s behavior from birth to death to the external environment is incalculable. The tissues, organs, proteins, RNA, and DNA of an organism change over time, and are also affected by the external environment and various behaviors. Similarly, existing data processing methods are difficult to make an effective analysis for such a large amount of biomedical data.

For the above reasons, analyzing the multidimensional and complex biomedical data needs to overcome many challenges, and biomedical research usually needs to choose a simplified and controllable research direction. For example, medical big data come from various sources including gene sequencing, protein sequencing, electron microscopy, optical microscopy, etc. At the molecular cell level, medical data are generated by clinical examination, medical imaging, surgery, autopsy, etc. At the tissue and organ level, medical big data are generated by environmental monitoring, smart cities, smart homes, wearable devices, etc. At the behavioral level, we can solve specific problems in a local dimension. However, the local dimension study based on medical imaging big data cuts in from the macroscopic tissue organ dimension, and can reflect the microscopic dimension information through image features. A patient’s CT image can contain 52,428,800 voxels, which contain between 1000 and 100,000 image features, while 1000 patients contain between 1 million and 100 million image feature big data. Such a large amount of data lay the foundation for artificial intelligence in medicine. In the spatial dimension, the relationship between medical image features and pathological gene analysis results is constructed from macro to micro. In the time dimension, the association between medical imaging features and treatment follow-up results was constructed from treatment to prognosis. This joint analysis can help achieve assisted precision diagnosis and treatment planning for patients. Molecular events represent the intrinsic molecular activities in the micro scope. The massive collection of molecular events in the micro scale is reflected in the macro scale as the medical image. The pathological changes are usually related to multiple abnormal molecular event. The massive amount of multiple abnormal molecular events would display as abnormal regions in the medical images. Though medical images are macro representation of the massive amount of multiple abnormal molecular events, it is still difficult to associate the macro images characteristics with the micro abnormal molecular events. Nevertheless, with the help of artificial intelligence, the potential information in the macro images can be mined in the approach of quantitative analysis. Then, the multiple molecular events can be analyzed by the quantitative information explored by artificial intelligence. In the earlier research, Michael Kuo et al. analyze the diversity of genetic and protein activities with the diverse radiographic features extracted from CT images [7]. They found that the combination of 28 radiographic features can reconstruct 78% of the global gene expression profiles. The radiographic features are also related with cell proliferation, liver synthetic function, and patient prognosis. The finding of Michael Kuo and his colleges indicate the substantial relation between the macro quantitative information and the multiple molecular events. Furthermore, Sun et al. extracted quantitative features from contrast-enhanced CT images to investigate the relationship with the CD8 cell tumour infiltration [8]. It was found that the quantitative features combined with RNA-seq genomic data can be used to assess the tumour-infiltrating CD8 cells and predict the response to anti-PD-1 or anti-PD-L1 immunotherapy. The macro image biomarker can directly reflect the micro changes of CD8 cell tumour infiltration, which is an important marker for tumor response. Sun’s work shows that the relationship between the macro images and the micro multiple molecular events cannot only be explored but also can be used to assess the clinical manifestation of the patient, such as the therapy response. Their findings are also validated in three independent cohorts, which indicating that the image biomarker is promising in predicting the immune phenotype of tumours. Similarly, Haruka et al. extracted quantitative image features capturing the shape, texture, and edge sharpness information of the glioblastoma [9]. They found three distinct phenotypic clusters are related to unique set of molecular signaling pathways, which were directly related to differential probabilities of survival. So, the macro image phenotypes are related to different prognosis by associating with the micro multiple molecular events. Thus, the distinct phenotypes of the images can provide as a noninvasive approach to stratify GBM patients for different targeted therapy and personalized treatment. In a recent study, Mu et al. analyze the relationship between the quantitative F-18-FDG-PET/CT image features and the EGFR mutation status, which is related to the longer progression free survival in patients treated with EGFR-TKIs [10]. They construct an EGFR deep learning score significantly and negatively associated with higher durable clinical benefit, reduced hyper progression, and longer PFS.

1.3. Concept of radiomics

In recent years, due to the advancement of storage and information technology, the medical image information of patients has been well preserved digitally. Compared with previous simple image processing based on small samples, the ever-growing number of medical images brings new research opportunities: (1) Based on a large amount of imaging data, a more accurate statistical model can be established to improve the level of diagnosis and detection of computer-aided diagnosis systems, so that its accuracy is comparable to human-level diagnosis; (2) More complex and expressive machine learning, pattern recognition, and statistical methods can play a better role with the big data, thus mining more potential laws and information from massive imaging data. The accumulation of medical image big data and the rapid development of artificial intelligence technology directly promote the new comprehensive analysis method in medicine.

Radiomics generally refers to the use of CT, PET, MRI, or ultrasound imaging as input data, extracting expressive features from massive image-based data, and then using machine learning or statistical models for quantitative analysis and prediction of diseases [4,11–13]. Compared with the traditional practice of using only manual viewing, radiomics analysis can extract high-dimensional features that are difficult to quantitatively describe in human visuals from massive data and correlate them with clinical and pathological information of patients to achieve the prediction of certain diseases or genes. Using advanced bioinformatics tools and machine learning methods, researchers are able to develop potential models that improve the prediction accuracy of diagnostic and prognostic approaches [4].

Radiomics is an emerging medical image analysis proposed by Lambin et al. [12] in 2012, which refers to the high-throughput extraction of a large number of image features from radiological images. In the same year, Kumar et al. supplemented the definition of radiomics to high-throughput extraction and analysis of a large number of advanced quantitative image features from CT, PET, and MRI [11], expanding the imaging modality and adding quantification analysis. In 2014, Aerts et al. published a breakthrough application in Nature Communications, pointing out the prognostic ability of radiomics [13], which caused widespread concern in the scientific research community. In general, radiomics refers to the extraction and analysis of highly representative quantitative image features from clinically large-scale imaging data, that is, using a large number of automated data feature description algorithms, the imaging data were transformed into high-dimensional extensible feature space, and the disease diagnosis and prediction of the case data were completed by comparing and analyzing the imaging data with clinical information. Furthermore, the radiomics analysis is based on the assumption that the quantitative image–based parameters have a certain correlation with the molecular phenotype or genotype of the tumor. Radiomics postprocesses the medical images that need to be collected in clinical diagnosis and treatment, extracts information that is difficult to see with the naked eye, and combines with other genomic data, metabolic data, and protein data to improve the efficacy prediction and prognosis of the tumor. Thus, personalized treatment of the patient is achieved.

1.4. Value of radiomics

Due to the pathological characteristics of different tumor types, they have different imaging performances. Different tumor image features also indicate completely different treatment methods, which directly affect the prognosis. At present, based on the subjective clinical experience of the doctor, prejudgment of the tumor is achieved through medical images. Based on the existing medical image feature analysis, some multidimensional texture features can accurately reflect the pathological information of the diseased tissue, which has important research value for the realization of individualized medical treatment. Therefore, a complete feature database can filter the subsequent key features and provide more comprehensive data support. The use of radiomics methods to assist in the predictive analysis of tumors and to give credible recommendations is of great practical significance.

Radiomics analysis mainly extracts and quantifies the features associated with a diagnosis from images. For example, in the CT images of tumors, there are differences in the tumors’ shape, size, and texture with different pathological grades. These features are often used by doctors as a basis for manual diagnosis, but the diagnostic results are subjective and relevant to the radiologists’ experience, which makes it difficult to realize objective and repeatable diagnosis results. However, in the analysis of radiomics, the features of these doctors’ qualitative descriptions can be quantitatively described by mathematical expressions from the perspective of images, thus providing an objective and repeatable diagnosis. Radiomics attempts to extract image features associated with diagnostic results through a large number of medical images, and directly analyzes results via the images, rather than just simple image processing. For example, researchers extracted high-dimensional image features by radiomics and found that these features were highly correlated with the prognosis survival of lung cancer, patients could be divided into different risk groups according to different values of these features, and different treatment plans could be implemented [13].

In addition to the use of imaging data, radiomics has introduced genetic analysis to improve diagnostic accuracy. In the traditional genetic analysis, whether a gene is mutated is determined by gene sequencing of tumor tissue sampled from a certain location of the tumor. However, due to the heterogeneity of the tumor, genetic mutations may occur in other parts of the tumor that are not sampled. Therefore, traditional genetic analysis may have sampling errors [4]. The mutated gene affects the growth of the tumor and is thus expressed in the imaging data [14]. Radiomic features can be extracted