Artificial Intelligence in Precision Health: From Concept to Applications

By Debmalya Barh

Artificial Intelligence in Precision Health: From Concept to Applications provides a readily available resource to understand artificial intelligence and its real time applications in precision medicine in practice. Written by experts from different countries and with diverse background, the content encompasses accessible knowledge easily understandable for non-specialists in computer sciences. The book discusses topics such as cognitive computing and emotional intelligence, big data analysis, clinical decision support systems, deep learning, personal omics, digital health, predictive models, prediction of epidemics, drug discovery, precision nutrition and fitness. Additionally, there is a section dedicated to discuss and analyze AI products related to precision healthcare already available.

This book is a valuable source for clinicians, healthcare workers, and researchers from diverse areas of biomedical field who may or may not have computational background and want to learn more about the innovative field of artificial intelligence for precision health.

• Provides computational approaches used in artificial intelligence easily understandable for non-computer specialists
• Gives know-how and real successful cases of artificial intelligence approaches in predictive models, modeling disease physiology, and public health surveillance
• Discusses the applicability of AI on multiple areas, such as drug discovery, clinical trials, radiology, surgery, patient care and clinical decision support

• Language: English
• Publisher: Academic Press
• Release date: Mar 4, 2020
• ISBN: 9780128173381

Book preview

Artificial Intelligence in Precision Health

From Concept to Applications

First Edition

Debmalya Barh

Table of Contents

Cover image

Title page

Copyright

Dedication

Contributors

Editor's biography

Preface

Section I: Artificial intelligence technologies

Chapter 1: Interpretable artificial intelligence: Closing the adoption gap in healthcare

Abstract

Acknowledgments

Artificial intelligence in healthcare

Why do we need interpretable intelligent systems in healthcare?

What does interpretability mean?

How to realize interpretability in intelligent systems?

Summary and road map for the future

Chapter 2: Artificial intelligence methods in computer-aided diagnostic tools and decision support analytics for clinical informatics

Abstract

Introduction

Artificial intelligence methods and applications

From concepts to applications

Conclusion

Chapter 3: Deep learning in precision medicine

Abstract

Acknowledgments

Introduction to deep learning

Hardware and software requirements for deep learning

ANN, CNN, and deep learning concepts

How deep learning transforms the study of human disease?

An example of deep learning implementation in medicine

Conclusion and future directions

Chapter 4: Machine learning systems and precision medicine: A conceptual and experimental approach to single individual statistics

Abstract

Introduction: Personalized medicine and precision medicine

First case study: Self-organizing maps (SOMs) and the case of quality-of-life scales

Second case study: Pick-and-squash tracking (PST) algorithm to cluster patients with and without Barrett disease

Third case study: Clustering of patients with and without myocardial infarction by means of auto-contractive map (auto-CM)

Fourth case study: Use of several different machine learning systems to classify the single individual allowing degree of confidence of the prediction

Discussion

Conclusions and future direction

Chapter 5: Machine learning in digital health, recent trends, and ongoing challenges

Abstract

Acknowledgments

Introduction

Training and testing: The machine learning pipeline

Machine learning algorithms

Machine learning in action: Exemplary tasks and case studies

Challenges and future work directions

Conclusion

Chapter 6: Data mining to transform clinical and translational research findings into precision health

Abstract

Introduction

Data mining strategies and techniques in clinical and translational research

Translating data mining to advance genomics in disease risk

Role of clinical research data warehousing in big data science

Integration of multiple data sources to advance precision health

Conclusion

Future direction

Section II: Applications of artificial intelligence in precision health

Chapter 7: Predictive models in precision medicine

Abstract

Introduction

Predictive analysis

Predictive modeling

Conclusions and future directions

Chapter 8: Deep neural networks for phenotype prediction in rare diseases: Inclusion body myositis: A case study

Abstract

Acknowledgments

Introduction

Case study-inclusion body myositis

Efficacy of the method

Conclusion

Chapter 9: Artificial intelligence for management of patients with intracranial neoplasms

Abstract

Introduction

Diagnosis

AI for treatment

AI for prognosis

Future challenges and directions

Conclusions

Chapter 10: Artificial intelligence to aid the detection of mood disorders

Abstract

Acknowledgment

Introduction

The case for AI-based objective diagnostic markers

Machine learning: A brief introduction

Data relating to mood disorders

Software platforms and smartphone applications

AI in action: Depression and bipolar disorder detection

Challenges and future work directions

Conclusion

Chapter 11: Use of artificial intelligence in Alzheimer’s disease detection

Abstract

Introduction

Artificial intelligence techniques in Alzheimer’s disease detection

Why artificial intelligence is important for AD

Conclusions and future directions

Chapter 12: Artificial intelligence to predict atheroma plaque vulnerability

Abstract

Acknowledgments

Introduction

Atheroma plaque vulnerability: Case of study

Machine learning techniques (MLT) as a helpful tool toward determination of plaque vulnerability

Discussion

Conclusions and future directions

Chapter 13: Artificial intelligence in cardiovascular medicine: Applications in the diagnosis of infarction and prognosis of heart failure

Abstract

Introduction

Summary of the main artificial intelligence algorithms

Application of artificial intelligence to the diagnosis of acute coronary syndromes and acute myocardial infarction

Artificial intelligence applied to the prognosis of heart failure

Conclusions and future directions

Chapter 14: Artificial intelligence-based decision support systems for diabetes

Abstract

Acknowledgments

Introduction

Diabetes management

Blood glucose prediction

Prediction of glycemic episodes

Insulin bolus calculators and advisory systems

Risk and patient stratification

Commercial systems

Conclusions

Future directions

Chapter 15: Clinical decision support systems to improve the diagnosis and management of respiratory diseases

Abstract

Introduction

A brief review of the machine learning methods used in respiratory care

Brief introduction to the methods of pulmonary function analysis

Artificial intelligence/machine learning methods to improve the pulmonary function analysis

Possible future directions

Conclusions and future directions

Chapter 16: Artificial intelligence in neuro, head, and neck surgery

Abstract

Introduction

Artificial intelligence in head and neck surgery

Artificial intelligence in neurosurgery

Conclusions and future directions

Chapter 17: Use of artificial intelligence in emergency medicine

Abstract

Medical informatics on emergency medicine

Artificial intelligence

Artificial intelligence and emergency medicine

Artificial intelligence studies in emergency medicine

Commercial precision systems used in emergency care

Conclusion and future aspects

Chapter 18: Use of artificial intelligence in infectious diseases

Abstract

Acknowledgments

Preamble on infectious diseases

Artificial intelligence in health care

The utilization of AI in infectious diseases

Improving the process

Conclusions and future perspectives

Chapter 19: Artificial intelligence techniques applied to patient care and monitoring

Abstract

Introduction

Patient care scenarios

Artificial intelligence approaches for health care

Data gathering and feature extraction

Data analysis

Feedback generation

Challenges and future directions

Chapter 20: Use of artificial intelligence in precision nutrition and fitness

Abstract

Introduction

How AI could help with precision fitness

Challenges and future perspectives

Section III: Precision systems in practice

Chapter 21: Artificial intelligence in precision health: Systems in practice

Abstract

Introduction

History and approaches of artificial intelligence in precision health

Applications of machine-learning approaches in precision health

Systems in place: AI-based commercial decision support systems in precision health practice

Other differential diagnosis generators

Other intelligent tools of interest

Conclusions and future directions

Index

Copyright

Academic Press is an imprint of Elsevier

125 London Wall, London EC2Y 5AS, United Kingdom

525 B Street, Suite 1650, San Diego, CA 92101, United States

50 Hampshire Street, 5th Floor, Cambridge, MA 02139, United States

The Boulevard, Langford Lane, Kidlington, Oxford OX5 1GB, United Kingdom

© 2020 Elsevier Inc. All rights reserved

No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or any information storage and retrieval system, without permission in writing from the publisher. Details on how to seek permission, further information about the Publisher’s permissions policies and our arrangements with organizations such as the Copyright Clearance Center and the Copyright Licensing Agency, can be found at our website: www.elsevier.com/permissions.

This book and the individual contributions contained in it are protected under copyright by the Publisher (other than as may be noted herein).

Notices

Knowledge and best practice in this field are constantly changing. As new research and experience broaden our understanding, changes in research methods, professional practices, or medical treatment may become necessary.

Practitioners and researchers must always rely on their own experience and knowledge in evaluating and using any information, methods, compounds, or experiments described herein. In using such information or methods they should be mindful of their own safety and the safety of others, including parties for whom they have a professional responsibility.

To the fullest extent of the law, neither the Publisher nor the authors, contributors, or editors, assume any liability for any injury and/or damage to persons or property as a matter of products liability, negligence or otherwise, or from any use or operation of any methods, products, instructions, or ideas contained in the material herein.

Library of Congress Cataloging-in-Publication Data

A catalog record for this book is available from the Library of Congress

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

ISBN 978-0-12-817133-2

For information on all Academic Press publications visit our website at https://www.elsevier.com/books-and-journals

Publisher: Stacy Masucci

Senior Acquisitions Editor: Rafael Teixeira

Editorial Project Manager: Sara Pianavilla

Production Project Manager: Debasish Ghosh

Cover Designer: Miles Hitchen

Typeset by SPi Global, India

Dedication

I dedicate this book to my mother Ms. Mamata Barh who is the soul and inspiration of my life.

Contributors

Said Agrebi     Yobitrust, Technopark El Gazala, Ariana, Tunisia

Omar Arnaout     Computational Neuroscience Outcome Center, Department of Neurosurgery, Brigham and Women's Hospital—Harvard Medical School, Boston, MA, United States

Göksu Bozdereli Berikol

Department of Emergency Medicine, Istanbul Bakirkoy Dr. Sadi Konuk Training and Research Hospital, Istanbul

Department of Neurosurgery, Karaman Public Hospital, Karaman, Turkey

Gürkan Berikol

Department of Emergency Medicine, Istanbul Bakirkoy Dr. Sadi Konuk Training and Research Hospital, Istanbul

Department of Neurosurgery, Karaman Public Hospital, Karaman, Turkey

Arthur Bertachi

Institute of Informatics and Applications, University of Girona, Girona, Spain

Federal University of Technology—Parana (UTFPR), Guarapuava, Brazil

Lyvia Biagi

Institute of Informatics and Applications, University of Girona, Girona, Spain

Federal University of Technology—Parana (UTFPR), Guarapuava, Brazil

Alessandro Boaro     Computational Neuroscience Outcome Center, Department of Neurosurgery, Brigham and Women's Hospital—Harvard Medical School, Boston, MA, United States

Durhasan Bozdereli     Department of Otorhinolaryngology, Mersin City Training and Research Hospital, Mersin, Turkey

Vitória Negri Braz     Pontifical Catholic University of Campinas—PUCCAMP, Campinas, Brazil

Massimo Buscema

Semeion Research Centre of Sciences of Communication, Rome, Italy

Department of Mathematical and Statistical Sciences, University of Colorado, Denver, CO, United States

Aletha Silva Caetano     Nove de Julho University—Uninove, São Paulo, Brazil

Francisco Edgar Castillo-Barrera     Autonomous University of San Luis Potosi, San Luis Potosi, Mexico

Ana Cernea     Group of Inverse Problems, Optimization and Machine Learning, Department of Mathematics, University of Oviedo, Oviedo, Spain

Myriam Cilla

Defence University Center (CUD), General Military Academy of Saragossa (AGM)

Aragón Institute for Engineering Research (I3A), University of Saragossa, Saragossa

CIBER-BBN, Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine, Zaragoza, Spain

Ivan Contreras     Institute of Informatics and Applications, University of Girona, Girona, Spain

Nicholas Cummins     ZD.B Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany

Giuliano Roberto da Silva     University José do Rosario Vellano—UNIFENAS, Alfenas, Brazil

Enrique J. deAndrés-Galiana

Group of Inverse Problems, Optimization and Machine Learning, Department of Mathematics

Computer Science Department, University of Oviedo, Oviedo, Spain

John Jaime Sprockel Díaz     Department of Internal Medicine, Hospital de San José—Fundación Universitaria de Ciencias de la Salud, Bogotá, Colombia

Jorge Luis Machado do Amaral     Department of Electronics and Telecommunications Engineering, State University of Rio de Janeiro, Rio de Janeiro, Brazil

Estela S. Estapé

Dr. Estela S. Estapé & Associates, Inc., Toa Baja

Research Center, San Juan Bautista School of Medicine, Caguas

University of Puerto Rico, Medical Sciences Campus, San Juan, Puerto Rico

Juan Luis Fernández-Martínez     Group of Inverse Problems, Optimization and Machine Learning, Department of Mathematics, University of Oviedo, Oviedo, Spain

Francisco Javier Fernández-Ovies     Group of Inverse Problems, Optimization and Machine Learning, Department of Mathematics, University of Oviedo, Oviedo, Spain

Ana Claudia Barbosa Honório Ferreira     University of Campinas—Unicamp, Campinas, Brazil

Danton Diego Ferreira     Federal University of Lavras—UFLA, Lavras, Brazil

Eric Fornaciari     Department of Mathematics of Computation, University of California, Los Angeles (UCLA), Los Angeles, CA, United States

Preetam Ghosh     Department of Computer Science, Virginia Commonwealth University, Richmond, VA, United States

Guillermo A. Gomez     Centre for Cancer Biology, SA Pathology and University of South Australia, Adelaide, SA, Australia

William B. Gormley     Computational Neuroscience Outcome Center, Department of Neurosurgery, Brigham and Women's Hospital—Harvard Medical School, Boston, MA, United States

Enzo Grossi

Villa Santa Maria Institute, Como

Semeion Research Centre of Sciences of Communication, Rome, Italy

Michael Hagan     Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, United States

Alexander F.C. Hulsbergen

Computational Neuroscience Outcome Center, Department of Neurosurgery, Brigham and Women's Hospital—Harvard Medical School, Boston, MA, United States

Department of Neurosurgery, Haaglanden Medical Center, The Hague, The Netherlands

Anushtha Kalia     Cluster Innovation Center, University of Delhi, New Delhi, India

Rishabh Kapoor     Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, United States

Vasileios Kavouridis     Computational Neuroscience Outcome Center, Department of Neurosurgery, Brigham and Women's Hospital—Harvard Medical School, Boston, MA, United States

Julia Klapper     ZD.B Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany

Andrzej Kloczkowski

Battelle Center of Mathematical Medicine, Nationwide Children’s Hospital

Department of Pediatrics, The Ohio State University, Columbus, OH, United States

Future Value Creation Research Center, Graduate School of Informatics, Nagoya University, Nagoya, Japan

Linda Laras

SJBSM Puerto Rico Health Justice Center & Division of Research & Statistics

San Juan Bautista School of Medicine, Caguas, Puerto Rico

Anis Larbi

Singapore Immunology Network, Agency for Science, Technology and Research

Department of Microbiology & Immunology, Yong Loo Lin School of Medicine, National University of Singapore, Singapore, Singapore

Saul Oswaldo Lugo Reyes     Immunodeficiencies Research Unit, National Institute of Pediatrics, Mexico City, Mexico

Adria Mallol-Ragolta     ZD.B Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany

Marco Mammi

Computational Neuroscience Outcome Center, Department of Neurosurgery, Brigham and Women's Hospital—Harvard Medical School, Boston, MA, United States

Neurosurgery Unit, Department of Neuroscience, University of Turin, Turin, Italy

Javier Martínez     International University of La Rioja, Logroño, Spain

Miguel Ángel Martínez

Aragón Institute for Engineering Research (I3A), University of Saragossa, Saragossa

CIBER-BBN, Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine, Zaragoza, Spain

Francisco Eduardo Martínez-Pérez     Autonomous University of San Luis Potosi, San Luis Potosi, Mexico

Giulia Massini     Semeion Research Centre of Sciences of Communication, Rome, Italy

Faith Matcham     Department of Psychological Medicine, Institute of Psychiatry, Psychology and Neuroscience, King’s College London, London, United Kingdom

Mary Helen Mays     Biomedical Informatics Core, Puerto Rico Clinical and Translational Research Consortium, University of Puerto Rico, Medical Sciences Campus, San Juan, Puerto Rico

Pedro Lopes de Melo     Biomedical Instrumentation Laboratory, Institute of Biology Roberto Alcantara Gomes and Laboratory of Clinical and Experimental Research in Vascular Biology, State University of Rio de Janeiro, Rio de Janeiro, Brazil

Maria Helena Baena de Moraes Lopes     University of Campinas—Unicamp, Campinas, Brazil

Aditya Nagori     CSIR-Institute of Genomics and Integrative Biology, New Delhi, India

Joseph J. Nalluri     Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, United States

Sandra Edith Nava-Muñoz     Autonomous University of San Luis Potosi, San Luis Potosi, Mexico

Jose I. Nunez-Varela     Autonomous University of San Luis Potosi, San Luis Potosi, Mexico

Carlos Ortíz     Office of Informatics and Educational Resources (OIRE), School of Health Professions, University of Puerto Rico, Medical Sciences Campus, San Juan, Puerto Rico

Silvia Oviedo     Institute of Informatics and Applications, University of Girona, Girona, Spain

Jatinder Palta     Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, United States

Estefanía Peña

Aragón Institute for Engineering Research (I3A), University of Saragossa, Saragossa

CIBER-BBN, Biomedical Research Networking Center in Bioengineering, Biomaterials and Nanomedicine, Zaragoza, Spain

Héctor Gerardo Pérez-González     Autonomous University of San Luis Potosi, San Luis Potosi, Mexico

Charrise Ramkissoon     Institute of Informatics and Applications, University of Girona, Girona, Spain

Zhao Ren     ZD.B Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany

Björn Schuller

ZD.B Chair of Embedded Intelligence for Health Care and Wellbeing, University of Augsburg, Augsburg, Germany

GLAM—Group on Language, Audio & Music, Imperial College London, London, United Kingdom

Joeky T. Senders

Computational Neuroscience Outcome Center, Department of Neurosurgery, Brigham and Women's Hospital—Harvard Medical School, Boston, MA, United States

Department of Neurosurgery, Haaglanden Medical Center, The Hague, The Netherlands

Tavpritesh Sethi     Indraprastha Institute of Information Technology, New Delhi, India

Arjun Sharma     Cluster Innovation Center, University of Delhi, New Delhi, India

William C. Sleeman, IV

Department of Computer Science

Department of Radiation Oncology, Virginia Commonwealth University, Richmond, VA, United States

Timothy R. Smith     Computational Neuroscience Outcome Center, Department of Neurosurgery, Brigham and Women's Hospital—Harvard Medical School, Boston, MA, United States

Abdulhamit Subasi     Information Systems Department, College of Engineering, Effat University, Jeddah, Saudi Arabia

Khajamoinuddin Syed     Department of Computer Science, Virginia Commonwealth University, Richmond, VA, United States

Josep Vehi

Institute of Informatics and Applications, University of Girona, Girona

Biomedical Research Networking Center in Diabetes and Associated Metabolic Disorders (CIBERDEM), Madrid, Spain

Amin Zadeh Shirazi     Centre for Cancer Biology, SA Pathology and University of South Australia, Adelaide, SA, Australia

Editor's biography

Dr. Debmalya Barh holds MSc (applied genetics), MTech (biotechnology), MPhil (biotechnology), PhD (biotechnology), PhD (bioinformatics), postdoc (bioinformatics), and PGDM (postgraduate in management) degrees. He is an honorary principal scientist at the Institute of Integrative Omics and Applied Biotechnology (IIOAB), India. Dr. Barh has blended both academic and industrial research for decades and is an expert in bioinformatics, integrative omics-based biomarker discovery, molecular diagnosis, and precision medicine in various complex human diseases and traits. He works with more than 400 scientists from more than 100 organizations in over 40 countries. Dr. Barh has published over 150 research publications, 35 book chapters, and has edited 24 cutting-edge, omics-related reference books published by Taylor & Francis, Elsevier, and Springer. He frequently reviews articles for Nature publications, Elsevier, AACR Journals, NAR, BMC journals, PLOS ONE, and Frontiers, to name a few. He has been recognized by Who’s Who in the World and Limca Book of Records for his significant contributions in managing advanced scientific research.

Preface

Debmalya Barh, Editor

The 21st century is the era of technological revolution. After the publication of first human genome sequence in 2003, a huge technological shift observed in genomics for the translation of human genome, and over the last few years computational approaches have made it possible to use the human genome for personalized health and wellness. Computational approaches such as genetic algorithm, support vector machines, artificial neural network, decision trees, natural language processing, hybrid methods, cognitive computing, etc., introduced artificial intelligence (AI) into health care, which became a game changer in personalized medicine or precision health care in terms of time, cost, and personalization.

Currently, several precision health-care approaches are based on AI. However, the end users like clinicians, practicing doctors, molecular diagnostic professionals, genetic counselors, clinics, and related industries are not much aware of these recent developments as the data are scattered in various domains. Further, few books are available, although some are old, which only deals with computational methods which are not helpful to the precision medicine professionals in decision-making.

To overcome these issues and fill the gap, the book Artificial Intelligence in Precision Health: From Concept to Applications is introduced, which can provide best possible understanding of AI and their real-time applications to non-computational biologists, clinicians, and precision health-care practitioners. This comprehensive reference book will (i) help in understanding the concept of AI, (ii) educate how the AI is used in precision medicine, and (iii) provide guidance to practicing doctors and clinicians, molecular diagnostic professionals, and genetic counselors on how to use AI-based products for decision-making in the field of precision health care.

The chapters (21 chapters) in this book are organized into three sections. Section I: Artificial intelligence technologies consists of six chapters. Chapter 1 by Prof. Sethi and colleagues presents Interpretable artificial intelligence: Closing the adoption gap in healthcare. In Chapter 2 Prof. Ghosh and his team have discussed in very simple way various Artificial intelligence methods in computer-aided diagnostic tools and decision support analytics for clinical informatics. Deep learning in precision medicine is described in Chapter 3 by Prof. Gomez and colleagues. In Chapter 4, Prof. Grossi’s team has presented Machine learning systems and precision medicine: A conceptual and experimental approach to single individual statistics. In Chapter 5, Dr. Cummins and colleagues have demonstrated Machine learning in digital health, recent trends, and ongoing challenges. Finally, in Chapter 6, Prof. Estape and team have summarized Data mining to transform clinical and translational research findings into precision health.

In Section II: Applications of artificial intelligence in precision health, 14 chapters are dedicated to AI applications in various diseases. In the first chapter in this section (Chapter 7), Prof. Berikol and Berikol have discussed Predictive models in precision medicine. Prof. Kloczkowski and colleagues in Chapter 8 have demonstrated "Deep neural networks for phenotype prediction in rare diseases. Prof. Mammi and team have overviewed the recent advancement in Artificial intelligence for management of patients with intracranial neoplasms in Chapter 9. In Chapter 10, Dr. Cummins and colleagues have discussed Artificial intelligence to aid the detection of mood disorders. In Chapter 11, Use of artificial intelligence in Alzheimer’s disease detection is summarized by Prof. Subasi. Artificial intelligence to predict atheroma plaque vulnerability by Prof. Cilla and team is discussed in Chapter 12. Prof. Sprockel Díaz in Chapter 13 has overviewed the applications of Artificial intelligence in cardiovascular medicine. In Chapter 14, Prof. Vehi and colleagues have demonstrated Artificial intelligence-based decision support systems for diabetes. Prof. Lopes de Melo and team in Chapter 15 have discussed about Clinical decision support systems to improve the diagnosis and management of respiratory diseases. Artificial intelligence in neuro, head, and neck surgery is reviewed by Prof. Berikol and colleagues in Chapter 16. In Chapter 17, Use of artificial intelligence in emergency medicine is discussed by Prof. Göksu Bozdereli Berikol and Prof. Gürkan Berikol. In Chapter 18, Use of artificial intelligence in infectious diseases is described by Prof. Agrebi and Prof. Larbi. Artificial intelligence techniques applied to patient care and monitoring is presented in Chapter 19 by Prof. Martínez-Pérez and colleagues. Finally, Chapter 20 of this section is dedicated to nutrition and fitness where Prof. Maria Helena and her colleagues have given a detail account of Use of artificial intelligence in precision nutrition and fitness."

Section III: Precision systems in practice (1) consists of one chapter (Chapter 21), which describes AI-based various commercial precision systems in practice along with their user guide. Prof. Saul Oswaldo Lugo Reyes has made an enormous effort to bring together almost all the available precision systems in this chapter.

Overall, in this book, we have tried to cover a broad spectrum of topics including the AI technologies and their applications in various diseases along with the commercial precision systems that are in practice. Further, 71 experts in the field from 17 countries (Australia, Brazil, Colombia, Germany, India, Italy, Japan, Mexico, The Netherlands, Puerto Rico, Saudi Arabia, Singapore, Spain, Tunisia, Turkey, United Kingdom, and United States) have contributed to this book. Therefore, we believe that the book will be of help to AI researchers and students, clinicians, molecular diagnostic and bio-computing professionals, and precision health industries in understanding the current status and future direction of AI-based precision health.

Section I

Artificial intelligence technologies

Chapter 1

Interpretable artificial intelligence: Closing the adoption gap in healthcare☆

Tavpritesh Sethia; Anushtha Kaliab; Arjun Sharmab; Aditya Nagoric    a Indraprastha Institute of Information Technology, New Delhi, India

b Cluster Innovation Center, University of Delhi, New Delhi, India

c CSIR-Institute of Genomics and Integrative Biology, New Delhi, India

☆ All authors contributed equally.

Abstract

Healthcare is poised to enter a new era of intelligent systems and enhanced human connection. The potential of artificial intelligence (AI) and machine learning (ML) technologies to assist decisions, optimize operations, and to free up quality human time will revolutionize how humans deliver and receive care. The success and expert-level performance of AI-based diagnostic systems have ushered in unprecedented optimism. However, there is a growing concern, about the ethics, safety, and equity in the delivery of care. The lack of clarity about its workings and the subsequent distrust has adversely affected the relationship of AI with the caregivers and receivers, thus precluding adoption. In this chapter we discuss interpretable AI as a road map toward building trusted AI that gets adopted in healthcare. The emphasis on the accuracy of AI alone needs to be revisited. Useful models will not only be accurate, but will also fulfill auxiliary criteria such as safety, fairness, privacy, accountability, and transparency. We will discuss why interpretability is key to adoption, what are its evolving meanings, and how it is realized across different classes of AI. We also compare some of these methods through concrete examples and a road map for the future for building toward the goal of trusted AI in healthcare, one paradigm at a time.

Keywords

Healthcare; Artificial intelligence; Machine learning; Trusted AI; Equity and inclusion

Acknowledgments

The authors acknowledge the inputs from Dr. Rakesh Lodha, Department of Pediatrics, All India Institute of Medical Sciences, New Delhi, India. The wiseR software and case study on predicting hemodynamic shock from thermal images using machine learning was supported by the Wellcome Trust/DBT India Alliance Fellowship IA/CPHE/14/1/501504 awarded to Tavpritesh Sethi.

Artificial intelligence in healthcare

Artificial intelligence (AI) has become ubiquitous in all sectors of industry including retail, finance, automotive, transportation, energy, and manufacturing and is deemed as the new electricity (Lynch, 2017). It is expected to boost the global general data protection (GDP) by $15.7 trillion by 2030, and although healthcare has been a laggard in the adoption of technology, AI is expected to be one of the central areas contributing to the next generation of healthcare (Rao and Verweij, 2017). The triumph of AI over the human mind in games such as Chess (Campbell et al., 2002) and Go (Silver et al., 2016) has created unprecedented enthusiasm about applying similar technology to relearn medicine from data and to create learning health systems. However, healthcare is much more complex than board games and it is crucial for healthcare practitioners to understand the definition, potential, limitations, and evaluation metrics that could make AI a bedside reality. Newcomers to the field and healthcare practitioners are often confused because of the lack of a clear definition of AI and the substantial overlap between the related areas of machine learning (ML) and statistical learning. The overlap has resulted from the confluence of computer science and statistics that has contributed to the growth of AI over many years. This was captured succinctly by Tom Mitchell as, Over the past 50 years, the study of machine learning has grown from the efforts of a handful of computer engineers exploring whether computers could learn to play games, and a field of statistics that largely ignored computational considerations, to a broad discipline that has produced fundamental statistical-computational theories of learning processes (Mitchell, 2006). The confusion about the definition of AI gets further amplified in healthcare because of the lack of mathematical training in clinical curricula and the late adoption of computational technology into medicine. For the purpose of this chapter, we define AI as a broad discipline encompassing computational technologies that can recognize patterns in specific tasks, extend the learned patterns to previously unseen data, and recommend or take responsive action. Specifically, this chapter focuses on interpretable artificial intelligence as one of the crucial directions which could enable healthcare to leapfrog into Healthcare 3.0.

Like most other disciplines, AI in healthcare is driven by the need to automate rote processes and to ease the cognitive burden of decision-making. In the 1960s and 1970s, problem-solving programs such as Dendral (Lindsay et al., 1993) and MYCIN (Shortliffe and Buchanan, 1975) were developed to predict chemical structures responsible for the given mass-spectra data and to recommend antibiotics, respectively. The MYCIN program produced a probable list of bacteria from a series of yes or no questions and was powered by 600 expert-driven decision rules. In the 1980s and 1990s, expert-specified systems paved way to fuzzy set-theoretic (Adlassnig, 2008) models and Bayesian belief networks (Miller, 1994) where experts specified the causal structure of the network, and then the strengths of these causal relationships were learned directly from the data. Bayesian networks ruled this decade as an elegant way of formalizing diagnostic reasoning and causal inference that clinicians could immediately connect with (Reggia and Peng, 1987). Simultaneously, this decade saw the initial developments in artificial neural networks (ANNs) and their applications in diagnoses such as of myocardial infarction (Miller, 1994). The rapid growth in computing capabilities guided by Moore’s law enabled ML and AI algorithms that suddenly came within the reach of commodity hardware such as servers and desktop computers. These included ML algorithms such as the support vector machine (SVM), which continue to be the most widely used algorithm in healthcare AI (Jiang et al., 2017) decision trees and ensemble methods such as random forest (Breiman, 2001) and gradient boosting machines (GBMs). In the past decade, AI in healthcare has been enabled by big data from electronic medical records (EMR), electronic health records (EHR), and hospital information systems (HIS), largely due to the regulations enforcing this requirement. This has been instrumental in the training and applications of deep neural network (Deep Learning, DNN)-based approaches which started to gain industrial popularity in 2010. Nowadays, such DNNs can have hundreds of layers, millions of parameters, and specialized modules that mimic the organization of neural structures in the brain. Convolutional neural network (CNN), a flavor of DNNs, has become particularly popular because of its success with image classification tasks and is one of the most widely used DNN technology in healthcare (Jiang et al., 2017) including skin cancer (Esteva et al., 2017) diabetic retinopathy (Gulshan et al., 2016), and chest radiographs (Coudray et al., 2018). Subsequently, nonimage-based tasks accomplished with deep learning have also gathered momentum, the significant milestones being deep patient, an autoencoder-based representation learning from EHR data on 700,000 patients (Miotto et al., 2016) and FDA clearance for a robotic device for clot removal (Thomas, 2018). However, the rapid growth and penetration of AI into healthcare has also created potential concerns in fairness, accountability, transparency, and equity, challenges which implore us to take a step back and consider the unmet need for interpretable AI, the central theme of this chapter.

Why do we need interpretable intelligent systems in healthcare?

Before diving into the what and how of interpretable AI, this section takes a step back to ask why interpretable AI is expected to fulfill an unmet need that has been preventing pervasive AI in healthcare, why we think interpretability to be a keystone in this transition, and what are the forces responsible for creating this need.

Right to explanation and the regulatory landscape

The recent enforcement of the European Union General Data Protection Regulation (EUGDPR—Information Portal, 2019) is one of the major forces behind model interpretability research as it provides the "right to an explanation" to subjects, that is, patients. Specifically, any automated decision-making system that operates on human subject data is now required to provide the meaningful information about the logic involved… in a concise, transparent, intelligible and easily accessible form, using clear, and plain language. Thus, as intelligence systems become increasingly accountable, the road ahead for AI in healthcare under this regulatory landscape will inevitably require explanation and interpretation as crucial components (Doshi-Velez et al., 2017).

Medicine as a quest for why

Historically, medicine has evolved as the science of diagnostic and prognostic reasoning. With an increasing number of AI applications and research outcomes claiming value, clinicians are not only taking notice but also questioning why an automated system took a decision. This quest for why is intricately linked to causality (Goldberg, 2018), a key component that enables explainable reasoning. However, causality continues to be a notoriously difficult challenge to tackle, both at philosophical and technical levels and is one of the most exciting areas of research in data science and AI.

The need for a culture of AI-assisted healthcare

Jim Collins has likened the evolution of a new strategy to a flywheel (Collins, 2019) that slowly gains momentum and after a certain limit leads to a tangential catapulting into the new normal. While healthcare has been a laggard in the adoption of technology, on the positive side, this has created a huge unmet need for urgent and disruptive changes. Healthcare Reform and the Affordable Care Act (Blumenthal et al., 2015) in the United States provided the initial push with the requirement of EMR, but it is the pull generated by conversion of data to knowledge which is expected to power the flywheel. This change is already happening and evident as a cultural shift toward seeing AI as an integral component of healthcare and explainability as an integral component of AI. Explainability creates shared meaning which is critical for adoption in a complex area such as healthcare. The social aspects of explanation are beyond the scope of this chapter, for which the interested reader can refer to the excellent work of Tim Miller (Miller, 2019).

Adoption in clinical decision-making

Adoption is a natural outcome of a culture change. Explainable AI is key to early adoption in any industry (Ben Lorica, 2019) and the culture of sharing models and insights will catalyze this further step. The existing culture of scientific discovery in medicine has created in-depth knowledge of disease mechanisms which provides a strong baseline for clinicians to accept or reject the models, thus highlighting the importance of explainability.

Relevance in the marketplace

Finally, interpretability would catalyze the transition from healthcare providers to the stakeholders in the marketplace, including the payers and the patients themselves. It is noteworthy that this is not only desirable, but also a necessary step in the context of GDPR which empowers the subject with the right to ask for explanation.

What does interpretability mean?

If interpretability is so essential, why is it not evaluated as a part of standard modeling practice along with indicators such as accuracy, precision, recall, and F1-scores? The answer lies in the challenge to formalize and quantify the concepts of interpretability and explainability. The definition of interpretability is an area of active deliberation because of increasing evidence that accuracy is just one of the indicators that defines the usefulness of a model and other indicators of model safety, ethics, and transparency are equally important. This need is especially acute in case of AI for healthcare and social good because of the high potential of AI to further widen the healthcare inequities and new AI strategies for mitigation of these challenges are needed (Sethi et al., 2018). However, in order to use interpretability as a metric of model performance, it has to be clearly defined first. Various academic, military, and industry groups have defined interpretability through its attributes such as safety, privacy, fairness, accountability, transparency, causality, and explainability among many others. This multiplicity of attributes has led to mushrooming of names such as Glassbox AI (Voosen, 2017), Whitebox AI (Pei et al., 2019), FATML (Fair, Accountable, Transparent Machine Learning), and XAI (explainable AI), the last two being the more popular among others. Most of these approaches address the same core idea of making sense to a human and indeed Finale Doshi-Velez and Been Kim (Doshi-Velez and Kim, 2017) define this attribute as the ability to explain or to present in understandable terms to a human. However, this leaves the challenge of measuring interpretability unaddressed, vis-à-vis the standard model performance indicators (MPI) such as accuracy, precision, and recall which are essentially numeric indices. Can we do better? One solution is to use model complexity as a proxy for interpretability, i.e., the higher the complexity, the lower the interpretability. In general, more complex models are less interpretable where Occam’s razor (Duignan, 2018) tells us to use the simpler of the models. However, the complexity of healthcare decisions often necessitates the use of complex models such as DNNs, which also tend to be more accurate and are inherently complex and non-interpretable. With hundreds of layers and millions of parameters, the weights of these models cannot be inspected by humans unlike those in simpler linear regression models. What is one to do in clinical decision-making scenarios where there is a concomitant need of higher accuracy and explainability to reliably identify and avoid fringe cases where the model’s decisions can raise ethical concerns? The rest of the chapter addresses this problem through a deep dive into the methods that are currently enabling interpretable intelligent systems, specifically in the context of healthcare.

How to realize interpretability in intelligent systems?

This section provides an overview of existing solutions for achieving interpretability in their increasing order of complexity. We illustrate these methods through clinically relevant case studies from our research and examples from the healthcare modeling community. At the same time, we will group these methods into a taxonomy based on the stage at which interpretability can be injected into a particular model.

Achieving interpretability by design

Medical knowledge has grown steadily over the last century with the formalization of study design and statistical methods that have yielded mechanistic underpinnings of health and disease. Therefore, there is a vast potential to cross-pollinate ML with medical insights and inject interpretable features into ML models. This approach is called feature engineering, and it fell out of favor with the advent of DNNs which could work directly with the raw data without the need for specified features. However, deep learning methods can sometimes lead to embarrassing (Barr, n.d.) and potentially blunderous decisions (AS and Smith, 2018). Therefore, there is a revived interest in human-centeredAI (Guszcza, 2018) and doctor-in-the-loop (Schuler et al., 2018) models. Here we review a case study where feature engineering combined with computer vision and AI allowed us to predict shock, a killer condition in children admitted to the pediatric intensive care unit (PICU), up to 12 h ahead of its detection through the gold standard shock index.

Case study: Predicting hemodynamic shock from thermal images using machine learning

About 10 million children die of shock every year all over the globe (Brierley et al., 2009). A large fraction of these children are under 5 years of age and live in developing countries. Shock is defined as the state of mismatch between oxygen demand and oxygen supply to body tissues. The most common causes of shock include highly prevalent conditions in these countries, including dengue, cholera, severe malaria, and trauma. Ironically, shock is also one of the most common reversible killers as its timely detection and management with fluids and vasopressors can prevent a large proportion of these deaths. In our research (Nagori et al., 2019) linked to this case study, we wanted to know, Can we predict shock in children admitted to an ICU using AI upon thermal images? We leveraged the dictum cold and clammy extremities indicate shock," a first-year medical-school knowledge for feature engineering an automated center-to-periphery difference (CPD) of skin surface temperature. This parameter was particularly attractive as it could be assessed noninvasively without contact with the child’s skin, and even remotely. We collected 539 thermal images on patients admitted to the pediatric ICU of a tertiary-care hospital in India. Earlier we had reported the establishment of the first pediatric ICU-based Big Data resource from India, the Sepsis Advanced Forecasting Engine for ICU (SAFE-ICU) (Sethi et al., 2017), which has warehoused > 350,000 patient-hours of continuous monitoring data at a 1-s resolution over the past 3 years. The demographic and vital sign information of children at the time of thermal imaging was extracted from this SAFE-ICU resource, whereas the computational details of the shock-prediction work (Nagori et al., 2019) are beyond the scope of this chapter. In short, the pipeline was as follows (Fig. 1). Data were partitioned into training and test sets. We used a stack of computer vision, ML, and longitudinal statistical models to perform shape-based detection of the abdomen and feet followed by calculation of CPD, a feature that was consumed by a longitudinal mixed-effects model to predict shock at 0, 3, 6, and 12 h of imaging. We used the current gold standard, the age-adjusted shock index for training our models. In contrast to our noninvasive, noncontact strategy, shock index using intra-arterial blood pressure is an invasive marker and requires puncturing of a major artery, and hence puts the children at a higher risk of developing life-threatening infections. The prediction was 75% accurate and sensitive for detecting and predicting shock up to 12 h in advance from the time of start of abnormal shock index, thus providing a head start in the absence of any such predictive indicators.

Fig. 1 An interpretable artificial intelligence (AI) model for predicting shock using feature engineering based on prior knowledge. This image is an unchanged version sourced from Nagori, A., Dhingra, L.S., Bhatnagar, A., Lodha, R., Sethi, T., 2019, Predicting hemodynamic shock from thermal images using machine learning, Sci. Rep. 9 (1), 91. doi:https://doi.org/10.1038/s41598-018-36,586-8 and is licensed under a Creative Commons Attribution 4.0 International License as per http://creativecommons.org/licenses/by/4.0/.

Achieving interpretability through inherently transparent models

This section discusses ML algorithms the outputs of which are directly interpretable by humans. Techniques like linear regression and logistic regression are inherently more transparent and are the most popular flavor of AI used in clinical studies. Since our objective is to introduce these methods to healthcare practitioners and biomedical researchers, we will illustrate these methods through a case study using a publicly available clinical dataset, the Cleveland Heart Disease Data (Detrano et al., 1989). The Cleveland dataset, downloaded from UCI, ML repository (Index /ml/machine-learning-databases/heart-disease, 1996), consists of 13 predictor (− dependent) variables and an outcome (response) variable num which originally had a range of 0–4 (representing number of major vessels). We transformed it into a binary variable where 0 represents no heart disease (num = 0) while 1 represents heart disease (num ≥ 1). The reader may note that the models presented are for the sake of illustrating interpretable ML methods and not optimized for research outputs as some of the details are missing in the accompanying documentation of the dataset. For the sake of completeness of the case study, the available description of each variable is given in Table 1. Instead of diving into the mathematical details for which the interested reader can refer to Tibshirani and Hastie (Hastie et al., 2009), we compare the salient features of various classes of interpretable models.

Table 1

The data consists of 303 patients with 139 patients without heart disease. Missing values in the variables ca and thal were imputed using replacement with the mode, a simplistic approach as data imputation is not the main theme of this chapter and 30% of the data was reserved as holdout data. All models were constructed using the Python programming language library sklearn (Pedregosa et al., 2012) and R (R Development Core Team, 2011).

Linear and logistic regression models

Linear regression is one of the easiest to interpret models where the weights (coefficients) and their sign can be directly interpreted by clinicians as effect size and direction, respectively. Due to the simplicity of linear relationships, it is easy to interpret the coefficient of a predictor as change in the outcome resulting from a unit change in the predictor. Similarly, the coefficient of a binary categorical variable indicates the change in the target variable; its value is changed from no to yes. Similarly, when the outcome variable is itself categorical and binary, i.e., a classification problem, logistic regression is commonly used where a unit change in the value of a predictor affects the odds of the outcome event as the exponential of the predictor’s coefficient. However, it is important to realize that this is a simplistic picture and in the real-world scenario; there may be no change in the value of the outcome variable when a predictor is changed as correlation does not imply causation. Furthermore, causality is one of the strongest proxies for explainability. On applying logistic regression to the Cleveland dataset, we achieved an accuracy of 79% on the test set. Other metrics were as follows: area under the curve (AUC) = 0.88, specificity = 0.86, sensitivity = 0.70, PPV = 0.80, NPV = 0.78. The top five features with maximum weights were asymptomatic for chest pain (cp_4), reversible defect on thallium scan (thal_7), and more than zero vessels blocked (ca_2, ca_1, and ca_3), respectively. Thus, changing the values of these features by unit amounts would affect the odds of heart disease the most. The final logistic regression equation obtained is as follows:

   (1)

Decision tree models (Quinlan, 1986)

These are a tree-like sequence of if-then-else rules, where the decision rules and their sequence is directly interpretable by clinicians, especially if the tree is small. Decision trees can be used for both classification (categorical outcomes) and regression (continuous outcomes) tasks. In contrast to decision rules defined by experts, AI techniques now allow these rules and trees to be learned directly from data and the outputs can be validated through a visual inspection of the tree. Internally, these models are learned by recursive splitting of data into subgroups based upon the purity (assessed by mathematical indices such as entropy and Gini impurity) of the resulting subgroups until the desired purity level is achieved with the final leaves of the decision tree. More importantly, the purity of subgroups created after a particular split is indicative of the feature importance of the predictor used in the respective decision rule. On the Cleveland dataset, a decision tree-based model achieved an accuracy of 72% on the test data. The splits obtained in this model are shown in Fig. 2.

Fig. 2 An illustrative decision tree trained directly from data from the Cleveland heart disease dataset. Color coding from light to dark blue represents an increasing probability of vessel blockage. The learned model captures that the first split induced by thallium scan, i.e., the probability of vessel blockage was zero if thallium scan was normal (3), chest pain was non-anginal or atypical (2, 3) and maximum heart rate achieved was > 160 and 19% of subjects from the training set fell into this category (bottom left block). On the other hand, a negative thallium test alone did not rule out blockage as the probability was as high as 91% if chest pain was anginal or absent, age was > 51, and exercise angina was present (bottom eighth block). The interpretability of sequential decision rules and the cut-off values for splits makes decision trees one of the most popular machine learning approaches in clinical decision-making.

Generalized additive models and partial dependence plots

Linear models, including generalized linear models (GLMs) do not allow for arbitrary nonlinear relationships to be captured (except for the simpler ones such as sigmoid dependence in logistic regression). What if a clinician wants to model a dataset which is known to have nonlinear dependencies? Generalized additive models (GAMs) (Hastie and Tibshirani, 1987) extend the GLM model where instead of modeling the outcome as a function of the weighted sum of predictors, a weighted sum of arbitrary functions of the predictor variables is derived instead. Therefore, it retains the interpretability associated with linear models with the additional advantage of visualization of the fitted nonlinear relationships using curves which can be inspected and interpreted by humans. These are known as partial dependence plots (PDPs) which essentially capture the relationship between a predictor variable and the outcome when other variables are kept at their mean. We trained a GAM on the Cleveland dataset essentially representing the following log odds of heart disease versus no heart disease:

   (2)

Here the left-hand side of the equation is the familiar log odds of vessel blockage and s1, …, sp. are smooth functions of the continuous predictor variables x1, …, xp. However, this alone is not interpretable because of the plausible nonlinear dependence of log odds on predictors. Therefore, PDPs (Friedman, 2001) are commonly used in conjunction with GAMs to visualize the partial dependence curves attributable to each of the features in the model. In the Cleveland dataset, we learned a GAM model using gam package (Hastie and Tibshirani, 1987) in R and visualized it with mgcViz package (Fasiolo et al., 2018). The accuracy of the learned model in predicting vessel blockage was 82.2% on the testing dataset. Further, PDP visualization showed that the dependence on continuous variables was linear in nature (Fig. 3), thus not necessitating the use of GAM in this dataset. A less complex model such as a GLM is sufficient in this dataset if interpretability and accuracy were the only evaluation criteria.

Fig. 3 GAM models followed by visualization using partial dependence plots (PDPs). The PDPs of variables (A) age, (B) thalach (maximum heart rate achieved), (C) chol (serum cholesterol), and (D) trestbps (resting blood pressure) indicate that relationships are mostly linear and expected as per disease physiology of coronary heart disease (CAD). Therefore, the use of a more complex model such as a GAM modeling might be an overkill in this case and Occam’s razor would recommend the use of a simpler GLM model if getting the visual trends and weights were the only desiderata being explored for interpretability. However, note that both GLM and GAM would fail to address causality, confounding effects and hence model safety , a key index of interpretable AI models.

Achieving model interpretability through post hoc methods

The next section takes a peek into the methods which are applicable to models that are neither interpretable by design nor interpretable by virtue of their intrinsic properties. Decisions suggested by these methods can be analyzed post hoc to understand why a particular decision was made by the model. Many of these techniques are model agnostic, i.e., can be applied to a large class of models including those which are intrinsically interpretable.

Feature importance

This is a useful method when AI practitioners want to see which predictors (features) played a crucial role in arriving at the decisions taken by the model. Feature importance can be described as the change in the model’s performance with the omission of a particular feature, the intuition being that the model’s accuracy should significantly decrease if an important feature were removed from the feature set. The same objective can also be achieved by the perturbation of values. We illustrate the latter approach for the calculation of feature importance for the decision tree model.

To find the important variables, error increase is computed by 1-AUC. The x-axis of the feature importance plot represents by what factor the error term 1-AUC increases when a feature is permuted. For instance, in Fig. 4, by permuting thal, the error increases by a factor of 1.75. An increase of error by a factor of one indicates that the variable has no effect on the outcome variable. According to this plot and method, thal, cp, and chol are the three most important features when a decision tree learner was used.

Fig. 4 Feature importance-based methods for model interpretability. Perturbation-based feature importance scores are shown in (A). Boruta (B) is a feature importance based method specifically designed for Random Forests and adds statistical rigor through significance testing of feature importance. Both methods agreed on the high importance of ca , thal, and cp for predicting vessel blockage as expected.

Boruta

This is a popular feature selection technique which is especially useful for the random forest algorithm (Breiman, 2001). Random forest is a black box ML method because it creates an ensemble of decision trees, which themselves are weak learners. However, the ensemble voted tree can be likened to a committee of experts evaluating slightly different aspects of the same data instance, hence yielding a robust solution which does not overfit. The tradeoff in constructing the forest is the loss of interpretability associated with decision trees. While Boruta does not make the Random Forest as interpretable as decision trees, it calculates the statistical significance of feature importance and Z-scores (Fig. 4B). This is a wrapper-based approach and uses the resampling methods to create shadow features and contrasts the importance of real features vis-à-vis the minimum, mean, and maximum importance achieved on the shadow features. On Cleveland dataset, Boruta was able to capture the significant importance (green) of ca (number of vessels found to be blocked on fluoroscopy) and thal (nature of defect seen on thallium scan). Note that this order is similar but not the same as the order of splits in the decision tree.

Shapley values (SHAP)

SHapley Additive exPlanations or SHAP (Lundberg and Lee, 2017) generalize the feature importance and builds on earlier approaches in combination with game theory and local explanations to provide a mathematically rigorous, unique, and accurate additive feature attribution. SHAP explains the output of a model as a sum of the effects that each feature on the final conditional expectation. Averaging over all possible feature orderings and game-theoretic proofs are utilized to guarantee a consistent solution. We illustrate SHAP through a strong method the XGBoost (Chen and Guestrin, 2016) that achieved an accuracy of 80.22% on the test of the Cleveland dataset. Fig. 5 illustrates the results where the absence of chest pain, i.e., cp_4 is the most important feature for this model reversibility of defect (thal_7), followed by ST depression (oldpeak). The difference in feature importance of this model illustrates two key things about interpretability, i.e., it is specific to the model in question and models may look at different aspects (features) to arrive at their predictions and (2) there is also a multiplicity of models, i.e., in the real-life scenario, many models can explain the data equally well. Choosing models in such scenarios would require a multiplicity of evaluation criteria as well, i.e., not laying excessive stress on model accuracy alone but also taking soft indices such as fairness, explainability, and safety into account.

Fig. 5 wiseR Bayesian Network (A) and Bayesian Decision Network (B) learned on Cleveland data transparently reveal the key influences that could improve diagnosis and highlight potential confounders in the data. wiseR ( Sethi and Maheshwari, 2018) is a free and open-source end-to-end artificial intelligence (AI)-based decision and policy enabling platform that allows direct deployment of intelligent systems in healthcare.

Surrogate trees

The driving intuition behind surrogate trees is that an interpretable model can inspect the predictions of a complex model and learn what was the black-box model thinking thus yielding an interpretable surrogate (Craven and Shavlik, 1996). This technique is model agnostic and requires access only to the data and the prediction function. Decision tree being a highly interpretable model is popularly used as a surrogate. However, surrogate models may not be very accurate as illustrated in our example where Random Forest has been used as the base model and decision tree as the surrogate. This resultant r² of 0.583 implies a poor approximation although both are tree-based methods.

Locally interpretable model-agnostic explanations (LIME)

The key intuition behind LIME (Craven and Shavlik, 1996) is that surrogate models may better approximate the black-box models locally, i.e., in the context of small perturbations, rather than globally. To do so, a modified dataset is generated by LIME through perturbations, corresponding to which predictions are obtained from the black box model. Next, an interpretable model such as a decision tree or a Lasso regression is trained on this new data, which approximates the black box model locally, rather than globally. The implementation of LIME has been illustrated in the deep learning section of the chapter.

Achieving interpretability through graphical models

This section discusses the use of probabilistic graphical models (PGMs), Bayesian decision networks (BDNs), and a case study on wiseR (Sethi and Maheshwari, 2018), an end-to-end BDN learning and deployment platform. Networks are intuitive representations of complex data and interactions that are pervasive in healthcare. Networks that rely on pairwise associations between predictors suffer from the problem of spurious associations and false positive predictions that may arise just by chance alone. PGMs, on the other hand, are an elegant approach that have an advantage of having an intuitive network representation (Fig. 5.) of a joint probabilistic model that fits the data as a whole rather than pairwise. BDNs extend Bayesian networks, a class of PGMs to decision-making settings through the use of decision theory. Our free and open-source platform, wiseR not only allows learning (AI), interactive querying (transparency), bootstrap evaluations (robustness), visualizations of decisions (safety), and deployment (accountability) but also enables knowledge discovery from inspection of the structure. The reason why "structure encodes meaning" in BDNs lies beyond the scope of this chapter, but briefly a BN structure hidden biases in the data such as confounder, mediator, and collider effects which are not available in any of the preceding methods. This attribute makes BDNs one of the most potent AI methods to mitigate ethical and social challenges (Sethi et al., 2018) that could arise from the use of indiscriminate and non-interpretable methods. Fig. 5 shows the key features of end-to-end wiseR-based analysis of the Cleveland data where expert knowledge was encoded as: the outcome variable can have incoming arrows but cannot have outgoing arrows. There were no other restrictions placed and the structure learning was carried out an odd number of times (101) to allow majority voting. The majority voted structure (Fig. 5B) clearly shows that the driving influences for diagnosis of blockages (num) are the number of major vessels visualized on fluoroscopy (ca) and reversibility of defect on thallium scan. These two variables isolate the diagnosis (num) variable from the remainder of the network. In one quick look, the network structure also indicates that there was no direct connection of blockage with cholesterol (chol) at least in this dataset, something that has also seen as the flat line in GAM and the nonsignificant effect size in GLM. Although those models could not have explained the reason behind this, wiseR BDN shows that gender, age, and the resting heart rate are blocking the flow of probabilistic influence from each of these to the outcome, i.e., are the confounding influences that do not allow the cholesterol node to exert influence when accounted for. Such transparency of probabilistic reasoning is unique to data-driven BDNs; thus, these are the most useful form of AI that can illuminate concerns about safety, transparency, usefulness, and accountability, criteria which remain unaddressed jointly by any other methods.

Achieving interpretability in deep neural networks

Deep learning has been one of the most revolutionary AI technologies of this century and is rapidly changing the landscape of AI as it takes away the human cognitive burden for hand engineering of features. Its ability to learn from large datasets and outperform classic ML models on various tasks has particularly made it popular. Healthcare is expected to be truly revolutionized by deep learning and imaging-based areas like radiology, pathology, dermatology, and ophthalmology are expected to be early adopters (Esteva et al., 2019). However, deep learning also has a deep problem of being a black box. Some deep learning architectures are esoterically designed and contain hundreds of layers and millions of parameters which are impossible to track by the human mind. This has led to major concerns for use of deep learning in areas such as healthcare where such systems could fail silently. Recent efforts have been addressing the need for interpretability. What is my DNN seeing? This question plagues every deep learning scientist and application developer because of the possibility of fringe cases where black-box models may falter and pose a serious threat to safety. For illustrative purposes, we repeat here the training and interpretability of the CheXNet (Rajpurkar et al., 2017) (Chou, 2018) architecture along with methods for interpretability using deep-viz-Keras (Anh, 2017), LIME (Ribeiro, 2019) on two outcomes (classes) of images, atelectasis and cardiomegaly. We utilized the openly available ChestX-ray14 dataset (Wang et al., 2017) which contains a total of 112,120 frontal-view chest radiograph images of 30,805 unique patients with 14 disease labels, namely atelectasis, cardiomegaly, effusion, infiltration, mass, nodule, pneumonia, pneumothorax, consolidation, edema, emphysema, fibrosis, pleural thickening, and hernia.

Taxonomy of interpretable deep learning methods

The methods used were saliency maps, integrated gradients, grad-CAM, and LIME. Fig. 6 illustrates that grad-CAM and integrated grad performs better than capturing the pathological areas in the thoracic cavity. We have deliberately delayed the mechanistic details behind the taxonomic classes of these methods (Fig. 7) as these